SFT-as-Context Mitigates Forgetting in Supervised Fine-Tuning.

arXiv:2610.11132v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) equips large language models (LLMs) with specialized capabilities, but often comes at the cost of forgetting the general capabilities of their parent models (i.e., the pretrained mo