No Pain, More Gain: Iterative Merging for Effective Multi-Teacher On-Policy Distillation.

arXiv:2609.34745v1 Announce Type: cross Abstract: Multi-teacher on-policy distillation (MOPD) combines independently developed domain teachers into a single student by distilling their predictions on student-generated samples. We study a setting where teachers