“Just because the model looks better on the benchmarks doesn't mean it actually got better on your very specific use case.”
Model launches make headlines. Our Chief AI Scientist @jefrankle gets into the research, RL, evals, and tradeoffs behind them with @readtechnically.
"You never quite know what you're going to get when you push the frontier." But what does pushing the frontier actually look like day-to-day? What do AI researchers...actually do? On Ep 2 of Technically Speaking, @jefrankle (Chief AI Scientist @databricks) & @meganreyno get into it: → How he runs his team: "If I don't have numbers, I don't believe you." → Why you should ship before you feel ready: "you need to ship the best of what you have at any given time" → Who he hires: "people who are never quite satisfied" 🎬 Watch the full episode: https://youtu.be/Qz58c13N7ms Presented by our friends at @Railway. Stay tuned for some of our favorite moments...
Databricks · x.comPublished
Original title: “Just because the model looks better on the benchmarks doesn't mean it actually got better on your very specific use case.”
