JudgeProfile: Understanding and Steering Subjectivity in LLM Judges.

arXiv:2609.36705v2 Announce Type: replace Abstract: LLM judges are inherently subjective, often favoring different responses in pairwise comparison when neither option is objectively wrong. To study this subjectivity, we introduce JudgeProfile, a framework tha