World Properties without World Models: Distributional Associations and the Interpretation of Decoding Results from Language Models.

arXiv:2603.04317v2 Announce Type: replace-cross Abstract: A growing literature shows that variables can be linearly decoded from the activations of large language models (LLMs). These range from properties of the world, such as the locations of cities and the