Estimating and Orthogonalizing Unknown Pre-training Gradients for Continual Fine-tuning of Large Language Models.

arXiv:2609.30935v1 Announce Type: cross Abstract: Continual fine-tuning is essential for large language models (LLMs) to dynamically adapt to real-world environments, yet it inevitably suffers from catastrophic forgetting, particularly the performance degradat