BA-DPO: Bias-Adjusted Direct Preference Optimization for Language Model Alignment.

arXiv:2609.35044v1 Announce Type: cross Abstract: Preference-based alignment methods such as Direct Preference Optimization (DPO) use pairwise preferences labeled by human annotators to fine-tune language models. However, annotators carry systematic biases tow