SparKV: Overhead-Aware KV Cache Loading for Efficient On-Device LLM Inference

arXiv:2604.21231v3 Announce Type: replace-cross Abstract: Efficient inference for on-device Large Language Models (LLMs) remains challenging due to limited hardware resources and the high cost of the prefill stage, which processes the full input context to con

← Back to all developments · Signals are a development reported by more than one source. See the methodology.