DivMoE: Fine-Grained MoE Upcycling via Cross-Domain Expert Composition.

arXiv:2610.11317v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures have become essential for scaling large language models, with recent work demonstrating the benefits of fine-grained expert designs. Training such models from scratch is exp