Low-Bit N-Gram Embeddings for Language Models.

arXiv:2610.31646v1 Announce Type: cross Abstract: Many recent open-weight language models use n-gram embedding tables to improve performance. These tables store learned representations of short token sequences, such as pairs or triples of consecutive tokens, w