SN 1093: Tokens in the Stream - Why LLMs are inherently insecure and prompt injection will persist
Summary
Steve Gibson and Leo Laport explore the mechanics of AI token streams, explaining how "role confusion" fundamentally causes prompt injection vulnerabilities. They discuss the controversy surrounding AI model distillation, where competitors train new models using outputs from mature systems, and analyze Anthropic’s strategy to safely deploy its powerful Mythos 5 cybersecurity model through restricted backend integrations. The episode also features Bitwarden, highlighting how their Secrets Manager protects credentials from agentic AI tools that might otherwise exfiltrate sensitive data via pr…
Topics discussed
Introduction and episode preview
Discussion on role confusion and podcast context
Episode title and proxy agent credentials
Warning on multitasking and focus
AI-driven malware and ThreatLocker security
ThreatLocker awards and escalator joke
LLM training data and post-training labor
Model distillation and data contamination
Web financing models and information freedom
Anthropic's Claude Mythos 5 security expansion
Sponsor segment: Box enterprise AI
1Password secrets management for AI agents
Hawk Hunt phishing simulation tools
Introduction of Lauren Cohnfelder and PKI
LLM threat model and token stream analysis
Mechanics of LLM context and token prediction
Break and GuardSquare mobile security sponsor
Role confusion and prompt injection mechanics
Tag parsing failures and attack success rates
Material security sponsor segment
Research experiments on role perception
Evolution of roles and inherent LLM limits
Conclusion, GRC.com resources, and credits
Listen ad-free on Castria