AI brief
A LessWrong post identifies a confound in no-cot-bench related to the positioning of the prompt's key, and says correcting for it reduces GPT-6.1 Sol's no-cot reasoning depth by 16%.
Why it matters: The post states the effect likely grows as dependent depth increases, which it says matters for interpreting looped transformers.
Written by AI from LessWrong's published text.
Source
“TL;DR : We identify a confound in no-cot-bench related to the positioning of the prompt's "key." Correcting for this decreases GPT-6.1 Sol's no-cot reasoning depth by 16%, with the effect likely growing as dependent depth increases.”
Interpretation or community commentary rather than straight reporting.
Read original story ↗