CausalMix: Data Mixture as Causal Inference for Language Model Training.

arXiv:2607.01104v2 Announce Type: replace-cross Abstract: In Large Language Model (LLM) training, data mixing plays a pivotal role in determining model performance. Recent methods optimize mixture weights via proxy models, but they rely on the assumption of st