Can You Trust Your Preference Models? Stackelberg Games for Alignment.

arXiv:2610.15008v1 Announce Type: new Abstract: Language models are frequently aligned to human preferences via imperfect evaluators whose errors can be exploited during optimization. We train policies to search for comparisons on which an evaluator may revers