Behind Aggregate Scores: What Systematically Varying Arena-Hard's LLM Judge Reveals.

arXiv:2610.30512v1 Announce Type: cross Abstract: LLM judges are used to determine which of two answers is better. Trusting these verdicts requires knowing how repeatable they are and how much they depend on the judge's instructions. We show that similar aggre