MeshKV: A Network-on-Chip KV Cache Fabric for Scalable Transformer Decoding Accelerators.

arXiv:2609.19207v1 Announce Type: cross Abstract: Autoregressive transformer decoding is constrained by irregular key-value (KV) cache movement on tiled accelerators. Prior compression and DRAM-placement systems still concentrate traffic on centralized memory