A Testable Theory of Atomic Features.

arXiv:2610.05794v1 Announce Type: new Abstract: We develop and test a theory of language model representations in which there exist atomic features. Our main theoretical insight is that in such a model, sparse dictionaries (e.g., SAEs) of increasing size recov