Rethinking LoRA Initialization for Robust Asymmetric Learning Rates.

arXiv:2610.32142v1 Announce Type: cross Abstract: Low-Rank Adaptation (LoRA) is often highly sensitive to learning-rate (LR) choices. Recent empirical studies suggest that, once LRs are carefully tuned, vanilla LoRA can remain competitive with many proposed Lo