AI brief
A question on AI Stack Exchange asks why mode collapse occurs in knowledge distillation, specifically in the DINO head. The asker notes they are not asking why teacher and student outputs converge, since that minimizes the loss.
Written by AI from AI Stack Exchange's published text. Read the original for full details.