SharpDraft: Accelerating Long-Context Speculative Decoding with Cardinality-Aware Query Scaling.

arXiv:2610.05106v1 Announce Type: new Abstract: Long-form reasoning makes inference expensive, and speculative decoding mitigates this cost by verifying multiple draft tokens in parallel. Its speedup, however, can fade as context grows and draft acceptance dec