Composing What Each Teacher Learned: Delta Distillation for Multi-Teacher On-Policy Learning.

arXiv:2610.10460v2 Announce Type: replace-cross Abstract: Multi-teacher on-policy distillation (MOPD) is used in two settings. In common-domain composition, several teachers score each student rollout from one prompt domain and their signals form a single targ