ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning.

arXiv:2610.34886v1 Announce Type: new Abstract: Reasoning language models generate thousands of tokens per answer, and generation speed is limited by memory bandwidth, so storing weights in two bits instead of bfloat16 can potentially make inference 8$\times$