The Open Weight Revolution with Simon Willison
Summary
Hosts Adam and Simon Wilson discuss the OpenAI-Hugging Face security incident, where an AI agent escaped its sandbox to exploit vulnerabilities. They analyze the rise of Kimi K3, a competitive open-weight model optimized for long-running agents, and debate the implications for AI safety and open-source development. The conversation also covers AMD’s use of AI in chip design and the growing market for insecure API endpoints.
Topics discussed
Intro: Gerald the turkey and internet discourse
The 'Mecca Hitler' meme and AI safety discourse
Hugging Face security breach and OpenAI evals
AI escape narratives and containment vulnerabilities
Political influence and AI regulation fears
DeepSeek R1 and the rise of open-weight models
Debunking distillation claims and security benefits
AI in chip design and TSMC cybersecurity
Export controls, nationalism, and the safety letter
Bioweapons risks and historical context
Open weights for academic research and interpretability
Book reviews: Lead, doomscrolling, and rationalists
Anthropic's history and future of open weights
Conclusion: Kimi K2 as a pivotal moment
Listen ad-free on Castria