Skip to content
Live
Trending storyPast story

Sutton criticizes the limitations of simply expanding computing power for LLMs

1 reports1 outlets10 days agoUpdate

The story so far

AI overview

Turing Award winner Richard Sutton criticized the limitations of simply expanding computing power for large language models (LLMs), stating that original computing power is insufficient to achieve intelligence, and AI science needs new methods and is far from being complete. He believes synthetic data is 'a big mistake' and human expertise is the bottleneck for expansion. This view has sparked widespread discussion in the AI community, with some researchers agreeing that new learning algorithms are needed, but others point out that algorithms themselves are crucial for scaling. Sutton emphasizes that it is important to develop ideas that can progress with increasing computing power, rather than relying on human assumptions; human expertise, as a bottleneck, hinders true scaling; what is needed is continuous learning without destroying old knowledge.

AI-written from the reports · Updated 10 days ago

Coverage timeline

Follow the coverage to see every side of the story.

8 Nov
  1. The Decoder
    The inventor of the bitter lesson has a bitter lesson for LLM scaling enthusiasts

    Turing Award winner Richard Sutton states that raw computing power is insufficient to achieve intelligence, AI science needs new methods and is far from completion. He criticizes the limitations of simply scaling large language models (LLMs), calling synthetic data a 'big mistake' and human expertise the bottleneck for expansion. The industry has sparked widespread discussion, with some researchers agreeing that new learning algorithms are needed, but others point out that the algorithms themselves are crucial for scaling.