How to Loop MoE: Flatten the Experts, Untie the Attention.

arXiv:2609.35751v1 Announce Type: cross Abstract: Looped Transformers reuse one block of layers several times: by spending extra computation they push a model of fixed size further, and so use its parameters more fully; while sparse mixture-of-experts (MoE) mo