Rethinking Cross-Tokenizer On-Policy Distillation: From Alignment Coverage to Supervision Reliability.

arXiv:2610.08448v1 Announce Type: cross Abstract: On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student predictions requires alignment at both sequence and vocabula