Augmenting Threat Intel Analysis with Agents - Chris Wallis, Sai Kiran Uppu, Ramin Farassat - ASW #396
Summary
In this episode, hosts Mike Shima and John Kinsella interview Cy Karanupu, a senior security researcher at Adobe, who details how his team uses LLM agents to automate threat intelligence analysis and hunt across petabytes of enterprise data. Karanupu explains the architecture of their multi-agent harness, which relies on specific markdown role definitions, budget constraints, and human-written query examples to prevent hallucinations and optimize performance.
Topics discussed
Intro, Star Trek riff, and InfoSecWorld sponsor
Guest intro: Saik Kiranupu on LLMs in threat intel
Motivation: Offloading grunt work to agents
Agent architecture: Roles, budgets, and markdown files
Grounding, sourcing, and hallucination mitigation
Data scale: Petabytes of logs and event streams
Sponsor: OWASP GenAI Security Project
Comparison to coding agents and context limits
Extracting atomic indicators and TTPs from data
Query corpus: Teaching agents optimized queries
Human-in-the-loop for threat escalation decisions
Case study: Missed criticality escalation
Proactive developer education and threat modeling
Identity monitoring and permission management
Threat modeling as brainstorming with AI
The importance of helpful logging and visibility
Using LLMs to find missing log sources
Supply chain attacks and missing header logs
Sponsor: GuardSquare mobile app security
Memory-based attacks and CI/CD hardening
Improving agent prompts and avoiding loops
Current challenges: Budget overruns and loops
AppSec in 3 words: Secure software
Closing remarks for first guest
Sponsor: ThreatLocker zero-trust execution
New guest: AI-enabled pen testing with Intruder
The economics of traditional pen testing
Scope of the AI pen testing tool
Speed and volume: 65 critical vulns in 4 hours
Human time constraints vs. AI consistency
Model selection and token budget efficiency
Finding flaws in 'secure' integrations like SSO
Optimizing for false negatives and positives
Non-determinism and the Hugging Face incident
Why strict methodologies fail with LLMs
The consistency problem in AI pen testing
Integrating traditional tools with AI
Ingesting entire codebases for reasoning
Distinguishing bugs from vulnerabilities
Remediation capacity and triage
The paradox of AI writing and finding code
Partial fixes and scope of remediation
Reasoning agents to curb AI over-hyping
AI as a better pen tester than humans
New guest: Menlo Security on agent attack surface
Prompt injection as the new phishing
Data vs. command separation in agents
Hidden text and invisible prompt injections
Attack surface: MCP servers and browser sidebars
Browser agents and sidebar vulnerabilities
Speed of agents and the 'evil bit' problem
Intent detection and command/data separation
MARS: Menlo Agent Real-time Security
Enabling AI adoption with confidence
Customer strategies: Restricting agent internet access
Frameworks: OWASP, NIST, and EU AI regulations
Government mandates for agent monitoring
Scaling challenges in enterprise environments
Securing the browser sidebar first
Shadow cloud and agent discovery
Using AI to discover AI-enabled sites
Governance before agent rollout
Maturity curve: Identification and visibility
Allow-listing agent tasks and behaviors
30-day focus: Agent internet traffic
Closing remarks and resources
Listen ad-free on Castria