AI Evaluation Should Require Standardized Item-Level Data Releases.

arXiv:2604.03244v3 Announce Type: replace Abstract: This position paper argues that standardized item-level benchmark data should become the default infrastructure for AI evaluation. Current evaluations suffer from underspecified item selection, construct misa