On-Policy Self-Distillation with Teacher Evolution.

arXiv:2610.38637v1 Announce Type: cross Abstract: On-Policy Self-Distillation (OPSD), which aligns a student model's token-level distribution with that of its self-teacher prompted by privileged context, has emerged as a promising direction to mitigate reward