Google DeepMind Researchers Propose "Streams" as Path to Superintelligence

"Experiential data will eclipse the scale and quality of human-generated data," declare legendary AI researchers David Silver and Richard Sutton in a groundbreaking new paper that outlines how autonomous AI systems could develop capabilities "that surpass those possessed by any human."

The two Google DeepMind luminaries have proposed a revolutionary framework called "streams" that could fundamentally transform how AI systems learn and develop, End of Miles reports.

Learning from experience, not just data

The framework, detailed in their paper "Welcome to the Era of Experience," represents a significant departure from current AI development approaches that rely primarily on massive text datasets and human feedback. Instead, Silver and Sutton argue that truly advanced AI must learn through continuous interaction with the world.

"Powerful agents should have their own stream of experience that progresses, like humans, over a long time-scale," Silver and Sutton write in their paper

The DeepMind researchers, renowned for their work on AlphaZero and reinforcement learning, believe this approach will enable AI to move beyond the limitations of today's generative AI models, which are restricted to isolated question-answer interactions without the ability to develop long-term memory or adaptation.

Beyond human limitations

What makes the proposal particularly significant is how explicitly it positions "streams" as a pathway to artificial general intelligence (AGI) and potentially superintelligence. The researchers don't mince words about their belief that this approach could create AI systems with capabilities beyond human intelligence.

"This paradigm shift, accompanied by algorithmic advancements in RL [reinforcement learning], will unlock in many domains new capabilities that surpass those possessed by any human." The researchers state

Silver, who led the team that created AlphaZero, and Sutton, a Turing Award winner for his work on reinforcement learning, suggest that AI development has been hampered by an overreliance on human judgment, which they describe as imposing "an impenetrable ceiling on the agent's performance."

How streams work

Unlike current AI systems that primarily respond to human prompts, AI agents in the streams framework would continuously interact with environments, setting and pursuing long-term goals while receiving feedback through various real-world signals.

The AI experts explain that these systems would develop their own world models through interaction, continuously updating them based on real-world experience. This approach would allow AI to learn from a vast array of signals including "cost, error rates, hunger, productivity, health metrics, climate metrics, profit, sales," and countless other real-world metrics.

"Once agents become connected to the world through rich action and observation spaces, there will be no shortage of grounded signals to provide a basis for reward. In fact, the world abounds with quantities..." The paper explains

Already underway

The researchers suggest that the transition to experience-based AI has already begun, pointing to web-browsing AI agents like OpenAI's Deep Research as early examples of more autonomous systems that can "act independently in the world."

The Stanford experts believe their proposed framework will eventually lead to "a future profoundly different from anything we have seen before," while acknowledging significant risks including reduced human oversight as these systems operate autonomously over extended periods.

Their paper, part of a forthcoming book by MIT Press, signals a potential paradigm shift in how leading AI labs approach the development of increasingly powerful AI systems – one that explicitly aims to move beyond human cognitive limitations toward AI systems capable of self-discovering knowledge in ways that humans cannot.