Few-Shot Truly Benign DPO Attack for Jailbreaking LLMs.

arXiv:2605.10998v2 Announce Type: replace-cross Abstract: Fine-tuning APIs make frontier LLMs easy to customize, but they can also weaken safety alignment during fine-tuning. While prior work shows that benign supervised fine-tuning (SFT) can reduce refusal be