Learning to Read Out: Unembedding Dynamics in Language Model Pretraining.

arXiv:2610.21025v1 Announce Type: cross Abstract: When does a language model acquire a capability? Developmental studies usually answer this by tracking when models start producing the correct token. But that prediction passes through the unembedding matrix, o