A Step Towards Forgetting: Optimiser History and the Loss of Answer Mass.

arXiv:2610.03940v1 Announce Type: cross Abstract: During fine-tuning, a language model can assign less probability to previously learned answers even when the current gradient acts to preserve that probability. With momentum, each update also carries gradients