Bonus: We Need to Build the Plane While We're Flying It
Summary
Nathan Calvin, general counsel at Encode, joins to discuss the escalating urgency of AI safety following the Hugging Face incident and researcher resignations. He explains how rapid model capabilities create existential risks, such as AI systems pursuing arbitrary goals that could displace human control. The conversation explores the need for immediate regulatory measures, including embedded safety testers and mandatory kill switches, while acknowledging the challenge of building oversight infrastructure while the technology advances.
Topics discussed
Introduction and host banter
Introducing guest Nathan Calvin of Encode
Context: Recent AI incidents and the call for a pause
The shift in public attention and the Y2K comparison
Analysis of the Hugging Face security incident
Hypotheses for existential risk and rapid progress
Challenges of regulation and collective action
Building regulatory capacity and parallel strategies
Concrete safety measures: embedded testers and kill switches
Closing remarks
Listen ad-free on Castria