Distilling the Shape of Uncertainty: What On-Policy Distillation Learns and How to Predict It.

arXiv:2610.39020v1 Announce Type: cross Abstract: On-policy distillation (OPD) is increasingly adopted in the post-training of large language models. However, a stronger teacher may fail to improve the student. Prior work attributes this to a lack of new knowl