OpenRouter rebuilds provider routing to weigh speed and reliability alongside price
AIOpenRouter says its new provider routing algorithm picks endpoints by a real-time Routing Cost that combines effective price, waiting time and retry cost. In a one-week switchback experiment over 24 million requests, it cut TTFT p50 by 16.9%, raised throughput p50 by 16.3% and reduced requests hitting a 429 by 67%, at a 6.0% higher cost per K requests.
AIWhy it matters.
The post explains how the router weighs price, latency, throughput and failures together, with a switchback experiment showing the trade-off in cost against speed and reliability.
Source details
OpenRouter Blog · openrouter.aiPublished · added here
Original title: How we rebuilt provider routing for better, faster inference