Inside the mindset of AI safety workers
Summary
Marketplace Tech explores the mindset of AI researchers through anthropologist Spencer Kaplan, who studied the Bay Area tech community. The episode discusses how effective altruism and existential risk concerns shape workers' decisions, such as skipping savings or quitting jobs to maximize impact. Kaplan explains the intellectual milieu of group houses and online debates, noting that these fears stem from direct experience with AI models. The segment also clarifies how researchers calculate "p(doom)" probabilities to guide their actions.
Topics discussed
Sponsor: Tomorrow is Cure podcast promo
Intro: The threat of out-of-control AI
Introducing anthropologist Spencer Kaplan
AI workers' beliefs and lifestyle changes
Why researchers stay vs. quit: The agency debate
The concept of 'high agency' in tech
Jacob Coxon's departure and public impact
The intellectual milieu of AI researchers
Fieldwork: Group houses and salon discussions
Focus on the future of humanity and AI
Sponsor: Tomorrow is Cure podcast promo
Return to interview: Effective altruism intro
Origins and spread of effective altruism
Core values: Maximizing positive impact
Shift from present problems to future risks
Connection between EA and AI safety researchers
Proliferation of EA values in the tech community
The ethic of 'you could just do things'
Outsiders' misconceptions about AI workers
Mystifying machines and questions of intelligence
Real experience vs. science fiction narratives
Understanding P(Doom) probability estimates
Practicing probability estimation in EA classes
Conclusion and credits
Sponsor: Pioneers of AI podcast promo
Listen ad-free on Castria