Coding-Agent Benchmarks Should Match Their Users' Task Flows.

arXiv:2610.09633v2 Announce Type: replace-cross Abstract: The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engineers in JetBrains IDEs, which we call Production Ses