Can We Trust This Evaluation? Instance-Level Reliability of LLM-as-a-Judge.

arXiv:2610.14029v1 Announce Type: cross Abstract: LLM-as-a-judge evaluation commonly aggregates multiple judges' decisions into a majority vote, yet this panel decision process does not guarantee that the evaluation of an individual item is reliable. We propos