MixLID: Token-Level Language Identification in Mixed-Language Text.

arXiv:2610.16601v1 Announce Type: cross Abstract: Processing multilingual text requires both identifying a broad range of languages and distinguishing the language of each token in inputs containing multiple languages. We introduce MixLID, a token-level langua