Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch.

i14 trained and served Laplace 3B, a linear attention model, on AWS Trainium2 using native PyTorch. The model's recurrent state stays fixed at 29.5 MB per sequence regardless of context length, avoiding the growing key-value cache of transformer models.

Why it matters: It shows a linear attention model can be trained and served on Trainium2 with native PyTorch while avoiding memory growth with context length.

Written by AI from AWS Machine Learning Blog's published text.