On Repulsive and Attractive Teachers: Separating Correctness from Behavior in Self-Distillation.

arXiv:2609.21561v1 Announce Type: cross Abstract: On-policy self-distillation provides dense, token-level supervision by conditioning a model on privileged information and distilling the resulting teacher distribution back into the model. However, privileged i