Training an MoE LLM in MXFP4: How We Closed the Gap to BF16.

arXiv:2610.32229v2 Announce Type: replace Abstract: Training large language models (LLMs) in sub-8-bit arithmetic promises substantial gains in memory bandwidth and compute throughput, yet FP4 formats introduce quantization error severe enough to destabilize o