Nash-MOPD: Optimizing Task Trade-offs in Multi-Teacher On-Policy Distillation.

arXiv:2610.26124v1 Announce Type: cross Abstract: Multi-teacher on-policy distillation (MOPD) aggregates the capabilities of multiple domain-expert teachers into a single student model. However, all domains in MOPD share the student's finite budget of optimiza