Chollet says scaling multiplies generalization, but cleverer methods scale better

AIFrançois Chollet says data and compute scaling act as multipliers on an approach's generalization power, and that huge investment can make low-generalization methods look adequate. He argues AI researchers should build methods that can scale, since those without this multiplier cannot compete, while noting that a cleverer approach will scale better.

François Chollet

@fchollet

View on X

Data scaling and compute scaling are *multipliers* for the generalization power of an approach. Trillions of dollars in investment can make very inefficient, low-generalization approaches look adequate, but that doesn't make them the ultimate end-all-be-all of AI.

You should work on methods that *can* scale, because if you don't benefit from this multiplier you simply can't compete. But you should keep in mind scaling is only a multiplier: a cleverer approach will scale *better*.

Richard Sutton@RichardSSutton

Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO

View quoted post on X

Source details

François Chollet · x.comPublished Nov 7, 5:52 PM ET · added here 6:20 PM ET

Original title: Data scaling and compute scaling are *multipliers* for the generalization power of an approach. Trillions of dollars in investment can ma...

Topics

Data & training Expert opinion