JustFit: Just-in-Time State Management for Local LLM Serving.

arXiv:2609.17475v3 Announce Type: replace Abstract: Local agents need memory for model execution and working history. We present JustFit, an MLX runtime that coordinates their overlapping allocations: KVExec executes and checkpoints four-bit KV with bounded wo