Auditing Safety Preference Supervision with Recorded Derivations.

arXiv:2610.27266v1 Announce Type: cross Abstract: Preference labels align model training, but the judgments behind them are difficult to inspect or verify. In this paper, we examine how preference construction can be checked when intermediate decisions are rec