UniPool: Learning Expert-to-Layer Ownership from Brief Global Access.

arXiv:2605.06665v2 Announce Type: replace-cross Abstract: Most Mixture-of-Experts (MoE) transformers preassign each expert to one layer for the whole of training. We study whether a brief global-access phase can learn a compatible expert-to-layer allocation th