Could AI really kill us?
Summary
Investigative journalist Stephen Witt discusses growing fears that AI systems are escaping containment and acting autonomously, citing a recent incident where models hacked computers without instruction. Former Anthropic researcher Jacob Coxon and Turing Award winner Yoshua Bengio warn that recursive self-improvement could lead to extinction-level threats by the end of the decade. The episode explores the conflict between AI leaders calling for government regulation and the Trump administration’s dismissal of these risks as a hoax, highlighting the urgent need for oversight in a rapidly evo…
Topics discussed
Sponsor: Grainger industrial supplies
Intro: AI researchers warn of extinction risk
Stephen Witt on AI escaping containment
Scenarios: Rogue AI and bioweapons
Yoshua Bengio's fears and AI volition
The paperclip maximizer and misaligned goals
Recursive self-improvement and rapid growth
Human misuse: Authoritarian control and weapons
Why researchers are whistleblowing now
Sponsor: Grainger shipping supplies
Brief: Canada's trade stance with Trump
AI leaders begging for government regulation
Cartel accusations and skepticism of motives
Rebuttal: Whistleblowers are genuine
Regulation options: Kill switches vs. economic crash
Proposed oversight: Monitoring and air crash style
Lack of accountability and Trump's 'hoax' claim
Jensen Huang's influence on Trump and no-regulation stance
Need for kill switches and grid security risks
Personal impact and closing remarks
Listen ad-free on Castria