Higher-order pruning of experts in mixture-of-experts language models.

arXiv:2609.18916v3 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) language models suffer from large parameter counts, which create a significant memory bottleneck. Expert pruning is the most direct approach for reducing this parameter count, y