In-Context Learning Before Specialisation: Theory of Learning in a Solvable Topic Model.

arXiv:2610.21901v1 Announce Type: cross Abstract: Sequence models can generalise in fundamentally distinct ways: extrapolating structures seen in training, or using their prompt to adapt to novel tasks. The prevailing strategy determines how the model behaves