LinearARD: Linear-Memory Attention Distillation for RoPE Restoration.

arXiv:2604.00004v3 Announce Type: replace-cross Abstract: The extension of context windows in Large Language Models is typically facilitated by scaling positional encodings followed by Continued Pre-Training (CPT). While effective, this paradigm is notoriously