Airbnb shares when to fine-tune a 2B model for production classifiers

Everyone’s excited about decision models. When should you buy one, or train your own?

We fine-tuned a 2B model for a production classifier: frontier-level quality, 35ms, ~1% of the cost.

We’re sharing when we think it’s worth it, and when it isn’t: https://airbnb.tech/ai-ml/frontier-quality-at-1-of-the-cost-a-production-benchmark-of-llm-classifiers/