TPO: Tri-level Distributionally Robust Learning for OOD Direct Preference Optimization.

arXiv:2610.28229v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) minimizes empirical risk over observed preference comparisons, implicitly assuming that the training preference distribution matches the deployment preference distribution.