Rethinking Routing Exploration in Modern Mixture-of-Experts.

arXiv:2610.22353v1 Announce Type: cross Abstract: Mixture-of-experts (MoE) models have become an important approach for scaling model parameters. Modern MoE architectures increasingly combine two design choices: fine-grained expert segmentation and shared expe