FactorEngram: Factorized N-gram Memory with Basis-Level Gating for Language Models.

arXiv:2609.35578v1 Announce Type: cross Abstract: Lookup-based memory has been a promising way to scale the parameters of large language models (LLMs). It retrieves learned representations of local token patterns, such as n-grams, instead of reconstructing the