Skip to content
Trending storyDeveloping

Epoch AI lowers weakest models' ECI scores after removing regularization

1 article1 sourcesince Nov 4Last article 2h ago ·

Overview

AISummary of 1 article

Epoch AI says models weaker than GPT-4, with ECI scores below 125, dropped an average of 14 ECI points after a scoring change.

The company attributes this partly to removing regularization from ECI model scores, which had pulled all models toward the average.

The post, published on X by Epoch AI, does not give further detail on the size of the change for stronger models.

Written by AI from the articles below · updated Nov 4, 4:36 PM ET

Check the sources:

Latest developmentNov 4, 4:31 PM ET
Epoch AI says weakest models dropped 14 ECI points after scoring change

Article timeline

The articles in this story. Times are ET.

Nov 4
  1. Epoch AIOfficial
    Epoch AI says weakest models dropped 14 ECI points after scoring change

    AIEpoch AI says models weaker than GPT-4, with ECI below 125, dropped an average of 14 ECI points. The company attributes this partly to removing regularization, which had pulled all model ECI scores toward the average.

    Scatter plot of each model's ECI under the new (Bayesian) method against the previous (frequentist) method, for all 252 models, colored by number of benchmarks. Strong models sit on the diagonal, while many weak models fall well below it, by up to about 40 points.

Heat Trend

Not enough continuous observations to show a trend yet.