Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model.
← Back to all developments · Signals are a development reported by more than one source. See the methodology.