A Stronger Teacher Within: Strengthening MoE Models via On-Policy Bidirectional Distillation.

arXiv:2610.23185v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved Mixture-of-Experts (MoE) LLMs, but its outcome-level rewards provide only sparse supervision. On-Policy Distillation offe