Maintaining Benchmarks Against Increasingly Capable Agents: Detection and Remediation of Unearned Passes.

arXiv:2609.34262v1 Announce Type: new Abstract: Agentic benchmarks guide model selection and training. Yet an agent can pass a task without demonstrating the intended capability. Such outcomes constitute unearned passes; their proportion among all passes defin