How Much Information Can We Fit inside a Transformer?

arXiv:2610.34553v1 Announce Type: new Abstract: How efficiently can transformers store factual knowledge? The answer to this question has implications for both model design and understanding the limits of transformer-based systems. Prior studies observe a line