How to post-train on a surrogate: Envelope sampling mitigates reward hacking.

arXiv:2610.11281v1 Announce Type: cross Abstract: Large language models (LLMs) are commonly post-trained against LLM judges and other cheap surrogates because the true reward, such as human preference, is too expensive to query at scale. This practice often le