TopK-Guided: Adaptive, Budget-Aware Activation Sparsity for Efficient LLM Inference.

arXiv:2610.01763v1 Announce Type: new Abstract: Activation sparsity speeds up large language model (LLM) inference by setting unimportant activations to zero so that the corresponding computations can be skipped. Existing training-free methods, however, make d