SimOPD: Relational Knowledge Transfer for On-Policy Distillation.

arXiv:2610.23306v1 Announce Type: new Abstract: A key challenge in on-policy distillation (OPD) is extracting effective supervision from teacher knowledge. Supervision from token probabilities may underutilize internal knowledge, while direct hidden-state matc