Skip to content
View original post on X: Rohan PaulX· 29/100AI score29/100

RouterPlus pools team demand to win bulk inference token pricing

AISummary

RouterPlus groups demand from many teams into one large volume deal with a provider, so each team gets the bulk token rate on its share of traffic. Rohan Paul argues this could help small teams access bulk pricing without being large buyers, especially for popular open models. The service also tests models on a customer's real usage to find the cheapest model that performs well enough.

Post on XView on X
Rohan PaulVerified on X
@rohanpaul_ai

Inference pricing has a size penalty. The best per-token rates go to whoever can commit the most volume.

RouterPlus groups demand from many teams into one large deal with a provider, and each team pays that rate on its share of the traffic.

Large buyers have done this with cloud commits for years. Applied to tokens, I’d guess it works best for popular open models, where many teams already want the same weights served on the same kind of hardware.

For a small team, this is a route to bulk pricing that doesn’t require being a bulk buyer.
They also test models on your real usage and pick the one that's good enough at the lowest cost. Many teams pay for a top model on every task when a cheaper one could do most of the work.

Alok Bishoyi@alokbishoyi97
Getting your inference right shouldn't be hard. @useRouterPlus (YC F26) is live: the right model and provider for your workload, at the best price. check out : https://routerplus.com/capacity
View quoted post on X

Source: Rohan Paul · x.comPublished · added here