Between Spoilers and Guesswork: Avoiding Over-Specification in Coding Benchmarks.

arXiv:2610.22760v1 Announce Type: new Abstract: Benchmarks drive AI progress, but are hampered by task under-specification and over-specification. Task under-specification causes guesswork, where solutions have to make things up, or worse, resort to contaminat