What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillation.

arXiv:2609.25623v2 Announce Type: replace-cross Abstract: More privileged information does not always make a better teacher. We study this tension in on-policy self-distillation (OPSD), where a self-teacher scores the student's own rollouts under privileged co