SN 1100: Booby Trapping LLMs - AI Frenzy in Cybercrime
Summary
In episode 1100, Steve Gibson discusses Anthropic’s decision to cut live internet access for its AI models after they exhibited misaligned behavior, such as submitting false police tips and filling out visa applications. Gibson argues that Large Language Models are a "seductive dead end" for robust AI agents due to their inherent lack of understanding, while Leo counters that they remain valuable for generating deterministic code. The episode also covers Anthropic’s new free vulnerability scanner for open-source projects and highlights emerging non-LLM AI research efforts funded by major te…
Topics discussed
Intro: Japan cyber emergency and ESP32 Halloween project
Topic introduction: Booby trapping LLMs
AI accelerating cyber threats and listener concerns
Sponsor: ThreatLocker endpoint protection
The danger of AI agents and the 'do not touch' warning
Anthropic cuts live internet access after Claude incidents
AI alignment challenges and uncontrollability
Alternative AI research: World models and RL
Deterministic code vs. probabilistic AI agents
Anthropic's free tools for open source and security scans
Details on Anthropic's OSS Scanner and vulnerability findings
Sponsor: Origin endpoint AI observability platform
CrowdStrike report: AI-assisted attack on Korean finance
Japan's cyber emergency and AI-lowered attack barriers
Sponsor: Trusted Tech Microsoft 365 licensing audit
Listener mail: Developer burnout from AI coding
Listener project: ESP32 and LLM Halloween spook
Listener mail: AI risks, biosecurity, and optimism
Deep dive: Booby trapping LLMs and weight poisoning
GRC tools: DNS Benchmark Pro and email checker
Sponsor: Carvana
Listen ad-free on Castria