Airbnb shares guidance on fine-tuning production classifiers

Airbnb says a fine-tuned 2B model reached frontier-level quality for a production classifier at 35ms latency and about 1% of the cost.

In a post by Ahmad Al-Dahle, Airbnb shares when it thinks fine-tuning a model is worth doing and when it is not, drawing on a production benchmark of LLM classifiers. The full guidance is in an Airbnb Tech article titled "Frontier quality at 1% of the cost: a production benchmark of LLM classifiers."