Are We Measuring Scientific Intelligence? Rethinking the Evaluation of AI Scientists.

arXiv:2610.04915v1 Announce Type: new Abstract: AI agents can now carry out data-driven scientific analyses end to end, and benchmarks assess them by giving an agent a question and a dataset and scoring its final answer against a fixed key. These benchmarks as