i14 runs its Laplace 3B linear attention model on AWS Trainium2 with native PyTorch
AIi14 and AWS trained and served Laplace 3B, a 3B-parameter linear attention model, on one trn2.48xlarge instance using native PyTorch through TorchNeuron Native. The model keeps a 29.5 MB recurrent state per sequence regardless of context length, and training reached about 120,000 tokens per second.
AIWhy it matters.
The post shows how a linear attention model's fixed recurrent state changes serving memory and cost, with concrete Trainium2 training and inference numbers.
Source details
AWS Machine Learning Blog · Kevin Shabahang · aws.amazon.comPublished · added here
Original title: Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch