Rater State Bias in RLHF Preference Data: An Audit Framework.

arXiv:2607.16195v3 Announce Type: replace Abstract: We identify a structured confound in Reinforcement Learning from Human Feedback (RLHF). Pairwise preference labels are intended to reflect the compared outputs, but they may also reflect the rater's state dur