Query Expansion and Key Specialization in Transformer Attention Geometry.

arXiv:2609.34273v1 Announce Type: new Abstract: The projection of queries and keys are central to the attention mechanism in Transformer architectures. While they are mathematically symmetric, they play different roles in attention mechanisms. The question of