AI researchers debate how close we are to recursive self-improvement
Sep 11, 2026 · 1h 37m
More Technology podcasts · More Science podcasts
Summary
Dwarkesh Patel interviews Baron Millich, John Schulman, and Charlie O'Neill to debate whether AI will achieve rapid recursive self-improvement by 2036. They argue that while scaling current RL paradigms may hit asymptotic limits, the key bottleneck remains the "sim-to-real" gap and the difficulty of generalizing from verifiable benchmarks to open-ended scientific discovery. The guests discuss how Chinese labs leverage router data for distillation, challenging the idea that frontier labs hold a permanent advantage.
Topics discussed
Introduction of AI researcher guests
The AI capability paradox and diminishing returns
Recursive self-improvement and discontinuities
Simulations, chess Elo scores, and regulation
Next token prediction and inner loop optimization
Generalization from verifiable to open tasks
The role of human analysis and theory building
Defining objectives and human-in-the-loop
Sponsor segment: Antithesis testing tools
Model distillation and prompt distribution
Benchmarking vs realistic deployment data
Automating AI R&D and environment design
Scaling RL environments and meta-skills
Learning from deployment vs pre-training
Sponsor segment: Jane Street chip design
Hive-mind learning and real-world interaction
Cumulative vs non-stationary task distributions
AI taste, context windows, and specialization
Catastrophic forgetting and RL efficiency
Sponsor segment: Grogbot podcast editing
Environment creation bottlenecks and RL signal
Data vs recipe improvements in pre-training
Architecture, sparsity, and scaling laws
Parameter scaling and Chinchilla limits
RL signal-to-noise ratio and mid-training
Discrete phase transitions in model learning
RL impact on creativity and diversity
Rapid-fire predictions on AI progress speed
More from Dwarkesh Podcast
Listen ad-free on Castria