Databricks' Jonathan Frankle on benchmarks, evals and shipping models

Databricks Chief AI Scientist Jonathan Frankle says a model that scores better on benchmarks may not improve a specific user's use case.

He made the remarks in Episode 2 of the podcast Technically Speaking, hosted with Megan Reynolds, which covers research, reinforcement learning, evals and the tradeoffs behind model launches. In the episode, he says he does not believe claims made without numbers, and that teams should ship the best available version rather than wait until a model is fully ready.