FuTransformer: FP8 prefill and INT4 decode for Llama-style inference on Hopper.

arXiv:2610.23796v1 Announce Type: new Abstract: Transformer prefill reuses weights across tokens and is dominated by matrix multiplication and activation data movement; single-token decode has limited reuse and is dominated by weight and KV-cache reads. We pre