PJFP.com

Pursuit of Joy, Fulfillment, and Purpose

Ilya Sutskever on the “Age of Research”: Why Scaling Is No Longer Enough for AGI

Written by

PJFP

in

In a rare and revealing discussion on November 25, 2025, Ilya Sutskever sat down with Dwarkesh Patel to discuss the strategy behind his new company, Safe Superintelligence (SSI), and the fundamental shifts occurring in the field of AI.

TL;DW

Ilya Sutskever argues we have moved from the “Age of Scaling” (2020–2025) back to the “Age of Research.” While current models ace difficult benchmarks, they suffer from “jaggedness” and fail at basic generalization where humans excel. SSI is betting on finding a new technical paradigm—beyond just adding more compute to pre-training—to unlock true superintelligence, with a timeline estimated between 5 to 20 years.

Key Takeaways

  • The End of the Scaling Era: Scaling “sucked the air out of the room” for years. While compute is still vital, we have reached a point where simply adding more data/compute to the current recipe yields diminishing returns. We need new ideas.
  • The “Jaggedness” of AI: Models can solve PhD-level physics problems but fail to fix a simple coding bug without introducing a new one. This disconnect proves current generalization is fundamentally flawed compared to human learning.
  • SSI’s “Straight Shot” Strategy: Unlike competitors racing to release incremental products, SSI aims to stay private and focus purely on R&D until they crack safe superintelligence, though Ilya admits some incremental release may be necessary to demonstrate power to the public.
  • The 5-20 Year Timeline: Ilya predicts it will take 5 to 20 years to achieve a system that can learn as efficiently as a human and subsequently become superintelligent.
  • Neuralink++ as Equilibrium: In the very long run, to maintain relevance in a world of superintelligence, Ilya suggests humans may need to merge with AI (e.g., “Neuralink++”) to fully understand and participate in the AI’s decision-making.

Detailed Summary

1. The Generalization Gap: Humans vs. Models

A core theme of the conversation was the concept of generalization. Ilya highlighted a paradox: AI models are superhuman at “competitive programming” (because they’ve seen every problem exists) but lack the “it factor” to function as reliable engineers. He used the analogy of a student who memorizes 10,000 problems versus one who understands the underlying principles with only 100 hours of study. Current AIs are the former; they don’t actually learn the way humans do.

He pointed out that human robustness—like a teenager learning to drive in 10 hours—relies on a “value function” (often driven by emotion) that current Reinforcement Learning (RL) paradigms fail to capture efficiently.

2. From Scaling Back to Research

Ilya categorized the history of modern AI into eras:

  • 2012–2020: The Age of Research (Discovery of AlexNet, Transformers).
  • 2020–2025: The Age of Scaling (The consensus that “bigger is better”).
  • 2025 Onwards: The New Age of Research.

He argues that pre-training data is finite and we are hitting the limits of what the current “recipe” can do. The industry is now “scaling RL,” but without a fundamental breakthrough in how models learn and generalize, we won’t reach AGI. SSI is positioning itself to find that missing breakthrough.

3. Alignment and “Caring for Sentient Life”

When discussing safety, Ilya moved away from complex RLHF mechanics to a more philosophical “North Star.” He believes the safest path is to build an AI that has a robust, baked-in drive to “care for sentient life.”

He theorizes that it might be easier to align an AI to care about all sentient beings (rather than just humans) because the AI itself will eventually be sentient. He draws parallels to human evolution: just as evolution hard-coded social desires and empathy into our biology, we must find the equivalent “mathematical” way to hard-code this care into superintelligence.

4. The Future of SSI

Safe Superintelligence (SSI) is explicitly an “Age of Research” company. They are not interested in the “rat race” of releasing slightly better chatbots every few months. Ilya’s vision is to insulate the team from market pressures to focus on the “straight shot” to superintelligence. However, he conceded that demonstrating the AI’s power incrementally might be necessary to wake the world (and governments) up to the reality of what is coming.

Thoughts and Analysis

This interview marks a significant shift in the narrative of the AI frontier. For the last five years, the dominant strategy has been “scale is all you need.” For the godfather of modern AI to explicitly declare that era over—and that we are missing a fundamental piece of the puzzle regarding generalization—is a massive signal.

Ilya seems to be betting that the current crop of LLMs, while impressive, are essentially “memorization engines” rather than “reasoning engines.” His focus on the sample efficiency of human learning (how little data we need to learn a new skill) suggests that SSI is looking for a new architecture or training paradigm that mimics biological learning more closely than the brute-force statistical correlation of today’s Transformers.

Finally, his comment on Neuralink++ is striking. It suggests that in his view, the “alignment problem” might technically be unsolvable in a traditional sense (humans controlling gods), and the only stable long-term outcome is the merger of biological and digital intelligence.

More posts