The Role of Quantization in Loss Spikes.

arXiv:2610.38277v1 Announce Type: cross Abstract: Loss spikes in language model pretraining can waste substantial compute and derail otherwise healthy training runs. A primary driver of these instabilities is the growth and divergence of output logits in the f