Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM Inference.

arXiv:2610.39562v1 Announce Type: cross Abstract: Multi-turn conversations with multimodal large language models (MLLMs) reuse input context, but long multimodal inputs increase time to first token (TTFT). Query-aware hard pruning reduces prefill tokens but pe