The Curse of Multilinguality in Large Language Models: Capacity Dilution, Tokenizer Fertility, and Representation Interference
The Curse of Multilinguality in Large Language Models: Capacity Dilution, Tokenizer Fertility, and Representation Interference Training a single transformer foundation model to process dozens or hundreds of languages is one of the central goals of modern natural language processing. In theory, massive multilingual pre-training unlocks positive cross-lingual transfer: low-resource languages gain syntactic, factual, and reasoning capabilities from the rich supervision available in high-resource l
1 min