SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation.

arXiv:2606.16454v3 Announce Type: replace-cross Abstract: Low-Rank Adaptation (LoRA) enables efficient adaptation of large pretrained models to downstream tasks by parameterizing weight updates with low-rank matrices. In this paper, we investigate the limitati