Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models.

arXiv:2609.38025v1 Announce Type: cross Abstract: On-policy distillation (OPD) trains a student on its own generated responses using dense, token-level supervision from a stronger teacher. Vanilla OPD treats all teacher signals equally, assuming that the teach