Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators.

arXiv:2606.07874v2 Announce Type: replace Abstract: LLMs-as-judges are the primary way to evaluate safety at scale. Despite their importance, LLM-judges themselves are rarely evaluated beyond human agreement in simple, static benchmarks. We therefore investiga