Skip to content
Trending storyDeveloping

Together AI advises testing models on own workloads over public leaderboards

1 article1 sourcesince Nov 8Last article

Overview

AISummary of 1 article

Together AI says public benchmarks are useful for building a shortlist of models, but teams should decide what to ship by replaying the workloads they serve and running their own evals.

The company advises using leaderboards to narrow the field and live traffic to make the final choice.

Written by AI from the articles below · updated Nov 9, 4:15 PM ET

Check the sources:

Latest developmentNov 8, 11:00 PM ET
Together AI says to test models on your own workloads, not leaderboards alone

Article timeline

The articles in this story. Times are ET.

Nov 8
  1. Together AIOfficial
    Together AI says to test models on your own workloads, not leaderboards alone

    AITogether AI says public benchmarks help build a model shortlist, but the real test is replaying the workloads you serve and checking which models pass your own evals. It advises using leaderboards to shortlist and live traffic to decide what to ship.

    Image from @togethercompute's post

Heat Trend

Current comparable heat
6
Peak comparable heat
6Nov 9, 5:00 PM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.