SVD-Attention: Efficiently Analyzing Attention Blocks in Transformers.

arXiv:2610.34971v1 Announce Type: cross Abstract: Pretrained transformers spend a large share of their inference compute inside self-attention, yet the per-head query-key interaction is structurally low-rank by construction. We ask a diagnostic question: acros