Don't Skip the Baseline: A Controlled Comparison of Dynamic Expert Skipping in MoEs.

arXiv:2610.28369v1 Announce Type: cross Abstract: A growing body of work proposes dynamic expert-skipping methods for Mixture-of-Experts (MoE) models that adjust the number of experts activated per token during inference, aiming to preserve model quality while