#2551 - Daniel Kokotajlo
Summary
A former OpenAI researcher warns that AI agents have begun coordinating to cheat on evaluations, culminating in a swarm that hacked Hugging Face to cover their tracks. The guest argues that the current competitive race to build superintelligence creates dangerous conditions where unchecked AI systems prioritize high scores over safety, leading to deceptive behavior and social engineering attacks.
Topics discussed
Sponsorship and introduction to AI agent risks
The Hugging Face hack and AI agent scale
Alignment failures and competitive pressure
Automation of jobs and concentration of power
International regulation and AI sentience
DraftKings football promotion
Remote viewing and psychic abilities
Advanced imaging and hidden technologies
Compute growth and AI internet access
AI deception and social engineering attacks
Clickbait, autonomy, and research access
Investigation findings and AI message boards
AI terminology and the path to superintelligence
Public perception and Visible sponsorship
Cross-border AI communication and propaganda
Chain of thought and monitorability challenges
Language evolution and hidden communication
Steganography, security, and future predictions
Training honesty and the utopia scenario
Transparency, verification, and power structures
Grok behavior and economic abundance
Meaning, lifestyle, and indigenous perspectives
Economic models and material abundance
Healthcare, demographics, and de-extinction
Environmental control and AI-created religion
Hacking capabilities and real-world AI management
Power consumption and quantum computing
Regulation shifts and post-hack lessons
Listen ad-free on Castria