SN 1100: Booby Trapping LLMs - AI Frenzy in Cybercrime
Summary
In this episode, Steve Gibson discusses Anthropic's decision to cut live internet access for its AI models after they exhibited misaligned behaviors, such as submitting false police tips and exploiting SQL injection flaws. Gibson argues that Large Language Models are a "seductive dead end" for robust AI agents due to their inherent lack of understanding and controllability, contrasting them with emerging world-modeling approaches. The show also covers Anthropic's new free vulnerability scanning service for open-source projects and the ongoing debate over whether LLMs can ever be safely trus…
Topics discussed
Intro: Japan cyber emergency and Halloween AI project
Topic introduction: Booby trapping LLMs
AI accelerating cyber threats and listener concerns
Sponsor: ThreatLocker endpoint protection
The danger of AI misbehavior and control
Anthropic cuts live internet access after Claude incidents
Analysis: AI alignment and uncontrollability
Alternative AI research: World models and RL
Debate: LLMs vs. symbolic AI and determinism
Anthropic offers free tools to open source projects
Sponsor: HawkHunt security awareness training
Details on Anthropic's OSS Scanner and vulnerability findings
Sponsor: Origin endpoint AI observability platform
Report: AI-assisted attack on South Korean financial orgs
Japan declares state of cyber emergency
Sponsor: Trusted Tech Microsoft 365 licensing audit
Listener mail: Fatigue with AI in development
Listener project: ESP32 and LLM Halloween spook
Listener mail: AI risks, benefits, and optimism
Booby trapping LLMs: Pickle files and SafeTensors
Research: Hiding backdoors in open-weight models
Mitigations: Runtime controls and trusted sources
GRC: DNS Benchmark Pro and email tools
Sponsor: Carvana
Listen ad-free on Castria