Skip to content
Trending storyDeveloping

Lucas Beyer says engram overfitting is ordinary overparameterization, not a new problem

1 article1 sourcesince Oct 11Last article 2h ago ·

Overview

AISummary of 1 article

Lucas Beyer says the overfitting seen with engram is ordinary overfitting that appears whenever a model is over-parameterized, not a problem specific to engram.

In an X post, he adds that engram, MoE, and similar methods make this overfitting easier and cheaper to reach. He also says it is not necessarily bad, depending on the goal.

Written by AI from the articles below · updated Oct 11, 8:52 AM ET

Check the sources:

Article timeline

The articles in this story. Times are ET.

Oct 11
  1. Lucas Beyer (bl16)X
    Lucas Beyer says engram overfitting is ordinary overparameterization, not a new problem

    AILucas Beyer says the overfitting seen with engram is plain overfitting that appears whenever a model is over-parametrized, not something specific to engram. He adds that engram, MoE, and similar methods make it easier and cheaper to reach, and says it is not necessarily bad depending on the goal.

Heat trend

Current comparable heat
9
Peak comparable heat
10Oct 11, 9:00 AM ET
Change in past 24 hours
–

The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.