What Matters in On-Policy Distillation? A Perspective on Data Efficiency and Data Selection.

arXiv:2609.05198v2 Announce Type: replace Abstract: On-Policy Distillation (OPD) has emerged as a widely adopted post-training paradigm for enhancing large language models in reasoning domains. However, the data-centric mechanisms in OPD remain relatively unde