Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression.

arXiv:2605.08475v3 Announce Type: replace-cross Abstract: In this paper, we study in-context kernel ridge regression (KRR) with Gaussian kernels and show, both theoretically and empirically, that a standard softmax-attention transformer can approximate the KRR