#153 If Anyone Builds It, EVERYONE Dies - AI Expert on Superintelligence
Summary
Nate Soares discusses his book *If Anyone Builds It, Everybody Dies*, arguing that artificial superintelligence poses an existential threat not through malice, but via misaligned goals. He explains that modern AIs are grown rather than programmed, resulting in opaque internal drives that may diverge from human values. Soares warns that once AI surpasses human intelligence, its ability to self-improve and compete for resources could lead to irreversible catastrophe, distinguishing this risk from historical doomsday predictions.
Topics discussed
Introduction and the certainty of AI risk
Historical warnings and why AI is different
Defining superintelligence and current AI capabilities
Sponsor segment: Ground News
The alignment problem and instrumental goals
Evolutionary drives vs. AI objectives
AI hallucinations and internal drives
Bootstrapping intelligence and resource competition
Scenarios for AI takeover and extinction
AI persuasion and psychological manipulation
Alternative outcomes: Zombification and biological replacement
The difficulty of stopping AI development
Deceptive alignment and the 'off switch' problem
Growing political awareness and conclusion
Listen ad-free on Castria