CM-DPO: Constraint-Margin Direct Preference Optimization for LLM Planning.

arXiv:2610.09219v1 Announce Type: new Abstract: Direct Preference Optimization (DPO) treats all constraint violations equally: a $1 budget overshoot and a $1,000 overshoot induce the same training signal. It is also susceptible to length and style bias when pr