DTDD: Divergence-Triggered Dynamic Distillation for Reliable On-Policy Supervision.

arXiv:2610.19302v1 Announce Type: cross Abstract: On-policy distillation (OPD) provides dense token-level supervision on student rollouts, but this supervision can become unreliable when the student and teacher diverge, either because of a large initial policy