The A.I. Researcher Whose Rebellion Is Changing Everything
Summary
Jacob Coxon, a former Anthropic researcher, resigned and went viral warning that AI could pose an existential threat to humanity by the end of the decade. In this episode, he explains his fears, citing rapid model improvements and the "Hugging Face" hacking incident as evidence of dangerous, unpredictable AI behavior. His warning prompted a rare consensus among industry leaders, including CEOs of Anthropic and OpenAI, to call for a global slowdown in AI development. Coxon argues that while immediate risks like job loss are manageable, an AI takeover would be irreversible, urging for regulat…
Topics discussed
Sponsor: Teram and the AI memory wall
Intro: AI existential risk warnings
Context: Industry leaders call for AI slowdown
Interview start: Jacob Coxon's viral tweet
Early AI experiences: AlphaGo and GPT-3
Defining the AI singularity and societal impact
Life at OpenAI: Data curation and research
AI progress: Math Olympiad gold medal
Moving to Anthropic: Culture and internal essays
Reasons for leaving: Hugging Face attack and fear
The decision to go public with the warning
Sponsor: Teram and The Athletic Live
Scenario: How AI could pose an existential threat
Rebuttal: AI autonomy vs. human-directed tasks
Defense strategies: Using AI to protect against AI
Industry response: Why insiders are speaking out
Bias check: Is the messenger credible?
Immediate risks: Jobs, security, and adaptation
Ideal outcome: Regulation and slowing down
Closing: Sleep and future plans
News: Trump, Middle East, and US Open results
Listen ad-free on Castria