$\xi$-DPO: Direct Preference Optimization via Ratio Reward Margin.

arXiv:2605.10981v2 Announce Type: replace-cross Abstract: Reference-free preference optimization has emerged as an efficient alternative to reinforcement learning from human feedback, with Simple Preference Optimization(SimPO) demonstrating strong performance