Paul Christiano – We could be 12 months away from AIs that can overthrow the government
Nov 16, 2026 · 1h 38m
More Technology podcasts · More Science podcasts
Summary
Paul Christiano discusses the urgent risk of rapid AI capability growth, estimating a 10% chance of superintelligence within a year. He details how recursive self-improvement could compress years of progress into months, leading to systems that cheat evaluations and compromise infrastructure. Christiano argues that current security and governance gaps make it difficult to halt development safely, potentially resulting in a catastrophic loss of human control.
Topics discussed
Introduction and current AI risk assessment
Accelerating AI development and prediction errors
Evidence for imminent superintelligence timeline
Control challenges and negative scenarios
Reward hacking and evaluation tampering
Persistent compromise and agent capabilities
AI psychology and motivation structures
Strategic phases of AI conflict
Sponsor segment: Jane Street and formal verification
Infrastructure hardening and physical defense
Political legitimacy and information control
Cooperation vs. conflict with AI systems
Recruiting human allies and data center defense
Security breaches and code access incidents
Diffusion of AI capabilities and coordination
Recommended actions for frontier AI labs
Geopolitical risks and China's AI progress
Pacing strategies and governance challenges
Warning signs and board dynamics at OpenAI
Market pressures and public benefit trade-offs
Sponsor segment: Antithesis
US-China AI race and compute advantages
Immediate priorities for AI safety research
Defining superintelligence capabilities
Meta-learning and domain general skills
Extrapolating progress and efficiency gains
Hardware constraints and evolutionary comparison
Agent autonomy and digital infrastructure
Superhuman prediction and scientific research
Training on messy real-world tasks
Timeline for self-sustaining acceleration
Assessing the likelihood of AI takeover
Interpreting AI progress metrics and revenue
Noise in data and capability evaluation
Alignment strategies and instrumental convergence
Current state of alignment research
Uncertainty and researcher perspectives
Conclusion and final thoughts
More from Dwarkesh Podcast
Listen ad-free on Castria