Teaching a Moving Student: Rethinking the Curriculum of On-Policy Distillation.

arXiv:2609.25048v2 Announce Type: replace-cross Abstract: In on-policy distillation (OPD), the student determines which states receive teacher supervision. As its policy evolves, earlier response prefixes become less likely even though teacher-student disagree