#378 The Data Engine for AI with Ledion Bitincka, CTO at Cribl & Nikhil Mungel, Head of AI R&D at Cribl
Summary
Valadin, Nikhil, and Ledian from Cribl discuss how AI agents are increasing software complexity and telemetry costs. They explain how logs, metrics, and traces help monitor system health and security, while also enabling "software factories" to automate code generation. The episode covers the shift from knowledge work to judgment work, emphasizing the need for engineers to validate AI outputs and manage token costs through data-driven insights.
Topics discussed
Sponsor: DataCamp AI skills
Intro: AI, telemetry costs, and guest introductions
Anecdote: AI coding agent incident and inference costs
Rising operational complexity and security risks of AI code
Defining telemetry: logs, metrics, and traces
Data volume, sampling, and the value of debug logs
Analyzing traces: conversational and agent workflows
Key goals: security, reliability, and user experience
Business signals: sales insights and runtime self-optimization
Automating insights: LLM loops and the software factory
The software factory model: from ticket to proposed change
Shifting engineer roles: from coding to judgment and review
Managing agents: decomposition and avoiding bill shocks
Cost optimization and model selection strategies
Productivity advice: focus, perseverance, and shipping value
Measuring success: customer validation and feature adoption
Empowering users: building custom apps on telemetry data
The future of personalized software and SaaS
Recommendations: leadership books and hardware trends
Outro: GPU analytics for telemetry and closing remarks
Listen ad-free on Castria