Learning from a Thoughtful Teacher: Adaptive On-Policy Self-Distillation for Mathematical Reasoning.

arXiv:2609.32667v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a question-only student with token-level feedback from a teacher given training-only privileged information (PI). OPSD therefore provides dense, on-policy supervision, an