Robust Direct Preference Optimization under Preference Label-Flip Attacks.

arXiv:2610.24945v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) is widely used for offline Reinforcement Learning from Human Feedback. However, recent works have shown that DPO can be easily manipulated by adversarial preference label fl