Scaling Laws for Looped Mixture of Experts.

arXiv:2609.40316v1 Announce Type: cross Abstract: Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed parameters, while MoE sparsity expands total capacity at fixed