Lucas Beyer says engram overfitting is ordinary overparameterization, not a new problem
Overview
Lucas Beyer says the overfitting seen with engram is ordinary overfitting that appears whenever a model is over-parameterized, not a problem specific to engram.
In an X post, he adds that engram, MoE, and similar methods make this overfitting easier and cheaper to reach. He also says it is not necessarily bad, depending on the goal.
Written by AI from the articles below · updated Oct 11, 8:52 AM ET
Check the sources:
Article timeline
The articles in this story. Times are ET.
Lucas Beyer (bl16)@giffmanaXLucas Beyer says engram overfitting is ordinary overparameterization, not a new problemAILucas Beyer says the overfitting seen with engram is plain overfitting that appears whenever a model is over-parametrized, not something specific to engram. He adds that engram, MoE, and similar methods make it easier and cheaper to reach, and says it is not necessarily bad depending on the goal.
Heat trend
- Current comparable heat
- 9
- Peak comparable heat
- 10Oct 11, 9:00 AM ET
- Change in past 24 hours
- –
The trend compares only continuously observed participants, so its scope may be smaller than the current heat total. Hover or tap for hourly heat; use arrow keys with a keyboard.