SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference

The memory-bound nature of the decoding stage of large language model (LLM) inference incurs significant latency.

← Back to all developments · Signals are a development reported by more than one source. See the methodology.