The Reward Model Selection Crisis in Personalized Alignment.

arXiv:2512.23067v3 Announce Type: replace Abstract: Personalized alignment from preference data has focused primarily on improving personal reward model (RM) accuracy, with the implicit assumption that better preference ranking translates to better personalize