What Does the Rank Buy? A Spectral and Distributional Analysis of Low-Rank Adaptation.

arXiv:2609.32002v2 Announce Type: replace Abstract: The rank $r$ in LoRA is widely treated as a capacity control: a smaller rank is assumed to yield a simpler model that generalizes better. We show that, under hard per-factor norm budgets---the idealization of