From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs.

arXiv:2609.25655v1 Announce Type: cross Abstract: As large language models (LLMs) scale rapidly, dense full-parameter adaptation becomes increasingly expensive, motivating sparse and modular architectures such as Mixture-of-Experts (MoE) models. This shift rai