On the API side, we cut the work the Messages API does before your request reaches the model, which lowered time to first token. Requests are also no longer buffered on their way in, on top of a long list of smaller infra improvements.
Anthropic cuts Messages API pre-processing to lower time to first token
AISummary
Anthropic says it reduced the work the Messages API performs before a request reaches the model, which lowered time to first token. The company also stopped buffering incoming requests and made several smaller infrastructure improvements.
Post on XView on X
Source: ClaudeDevs · x.comPublished · added here