SN 1093: Tokens in the Stream - Why LLMs are inherently insecure and prompt injection will persist
Summary
Steve Gibson and Leo Laporte explore the technical roots of prompt injection, identifying "role confusion" as the fundamental flaw in how LLMs process mixed inputs. They discuss the controversy surrounding AI model distillation, where competitors train new models on proprietary outputs, and examine Anthropic’s strategy to safely deploy its powerful Claude Mythos 5 model for cybersecurity defense. The episode highlights the inherent security risks in current AI architectures and the ethical debates over data usage.
Topics discussed
Introduction: The inherent insecurity of LLMs
Tokens in the Stream and multitasking illusions
Model distillation and training on AI content
Anthropic's Claude Security and Mythos 5 expansion
Secret management for coding agents and .env risks
Loren Kohnfelder's analysis of LLM security flaws
How LLMs process tokens and role tags
Role confusion and prompt injection mechanics
Why tags fail: Style over structure in LLMs
Conclusion: The limits of AI security and show notes
Listen ad-free on Castria