# Databricks' Chief AI Scientist discusses model benchmarks, evals, and shipping on a podcast

> Original title: “Just because the model looks better on the benchmarks doesn't mean it actually got better on your very specific use case.”

- Source: Databricks
- Published: 2026-10-29T17:53:02.000Z
- Bad Influence: https://badinfluence.news/items/a4vrf2egflhazny0316v48ajq
- Original: https://x.com/databricks/status/2115864501690007888

## Summary

Databricks Chief AI Scientist Jonathan Frankle says a model that looks better on benchmarks may not improve a user's specific use case. He discusses research, reinforcement learning, evals, and tradeoffs behind model launches in Episode 2 of Technically Speaking, hosted with Megan Reynolds. In the episode, he says he does not believe claims without numbers and that teams should ship the best available version rather than wait until ready.

## Post

“Just because the model looks better on the benchmarks doesn't mean it actually got better on your very specific use case.”

Model launches make headlines. Our Chief AI Scientist @jefrankle gets into the research, RL, evals, and tradeoffs behind them with @readtechnically.

## Quoted post @readtechnically

> "You never quite know what you're going to get when you push the frontier."
> 
> But what does pushing the frontier actually look like day-to-day?  What do AI researchers...actually do?
> 
> On Ep 2 of Technically Speaking, @jefrankle (Chief AI Scientist @databricks) & @meganreyno get into it:
> 
> → How he runs his team: "If I don't have numbers, I don't believe you."
> 
> → Why you should ship before you feel ready: "you need to ship the best of what you have at any given time"
> 
> → Who he hires: "people who are never quite satisfied"
> 
> 🎬 Watch the full episode: https://youtu.be/Qz58c13N7ms
> 
> Presented by our friends at @Railway.  
> 
> Stay tuned for some of our favorite moments...

https://x.com/readtechnically/status/2115806300047098126