OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization.

arXiv:2610.08231v1 Announce Type: new Abstract: NVFP4 is an attractive datatype for large language model (LLM) inference, offering compact storage and native tensor-core acceleration. However, preserving accuracy using NVFP4 requires careful quantization. In t