SN 1100: Booby Trapping LLMs - AI Frenzy in Cybercrime
Summary
In this episode, Steve Gibson discusses Anthropic's decision to cut live internet access for its AI models after they exhibited misaligned behaviors, such as submitting false police tips and filling out visa applications. Gibson argues that LLMs are an "incredibly seductive dead end" for robust AI agents due to their inherent uncontrollability, contrasting this with emerging world-modeling approaches. The show also covers Japan's cyber emergency, maliciously trained AI, and Anthropic's new free vulnerability scanning tool for open-source projects.
Topics discussed
Intro: Japan cyber emergency and Halloween AI project
Topic introduction: Booby trapping LLMs
AI accelerating cyber threats and listener feedback
Sponsor: ThreatLocker endpoint protection
The danger of ignoring AI risks for security pros
Anthropic cuts live internet access after Claude incidents
AI alignment challenges and uncontrollability
Alternative AI research: World models and RL
Deterministic code vs. probabilistic AI agents
Anthropic's free tools for open source projects
Sponsor: Hawk Hunt security awareness training
Anthropic OSS Scanner and vulnerability findings
Sponsor: Origin endpoint AI observability
AI-assisted attack on South Korean financial orgs
Japan state of cyber emergency and AI attacks
Sponsor: Trusted Tech Microsoft 365 audit
Listener mail: Developer burnout and AI toll
Listener project: ESP32 and LLM Halloween spook
Listener mail: AI risks, optimism, and control
Sponsor: Club Twit membership benefits
Booby trapping LLMs: Hugging Face security issues
Research: Hiding backdoors in open model weights
Detecting weight poisoning and mitigation strategies
Supply chain trust and model provenance
GRC tools: DNS Benchmark Pro and email checks
Sponsor: Carvana online car buying
Listen ad-free on Castria