Mitigating Memorization In Language Models.

arXiv:2410.02159v3 Announce Type: replace-cross Abstract: Language models (LMs) can "memorize" information, i.e., encode training data in their weights in such a way that inference-time queries can lead to verbatim regurgitation of that data. This ability to e