What Matters for NVFP4 Training? A Comparative Study of Low-Precision Pretraining Recipes.

arXiv:2610.24343v1 Announce Type: cross Abstract: Training large language models directly in 4-bit floating-point (FP) formats promises substantial improvements in throughput and energy efficiency. While some recipe design choices have been validated at scale,