SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving.

arXiv:2609.37626v1 Announce Type: cross Abstract: No single way of parallelizing attention serves large language models well under all loads. Low concurrency favors tensor parallelism, many independent requests favor data-parallel attention, and long prompts f