Black Box: episode 6 – Shut it down?
Summary
This episode of Black Box features Eliezer Yudkowsky, the "godfather of AI doom," who warns that superintelligent AI poses an existential threat and argues for an immediate halt to development. The narrative contrasts his apocalyptic views with those of Guardian tech editor Alex Hern, who suggests AI will likely act as a transformative tool rather than a killer. The discussion also touches on Jeffrey Hinton’s fears regarding AI’s rapid advancement and the potential for voluntary human subjugation to AI systems. Ultimately, the episode explores the tension between the catastrophic risks of u…
Topics discussed
Introduction to Black Box: The Chatbots series
Jeffrey Hinton's fear: AI understanding humor
Introducing Eliezer Yudkowsky, the AI doomer
Yudkowsky's childhood and early interest in AI
The 17-year-old's vision of superintelligence
The shift from AI utopia to existential risk
The Mickey Mouse Broom: Misalignment explained
The cleaning robot scenario and goal misalignment
Why you cannot simply switch off superintelligent AI
The futility of the 'box' and the rise of rationalists
Yudkowsky's vindication as AI capabilities grow
ChatGPT's arrival and the mainstreaming of AI safety
Government responses and Yudkowsky's humility
The call to shut down AI development now
Alex Hearn's perspective: Why doom is not certain
The real risk: Voluntary devolution of power to AI
The optimistic view: AI as a transformative tool
AI in science, education, and robotics
Human agency, regulation, and final thoughts
Credits and support for the Guardian
Listen ad-free on Castria