One Head Away from Language Confusion: Single Attention Heads Across Model Families.

arXiv:2610.25026v1 Announce Type: cross Abstract: Large language models sometimes reply in an unexpected language, a failure known as language confusion. We ablate individual attention heads in 12 model checkpoints spanning seven model series, using FLORES-200