Programmatically Developing LLM Prompts With DSPy
Summary
Brett Kennedy returns to discuss his new book, *Building LLM Applications with DSPy*, which advocates for "prompt programming" over manual prompt engineering. He explains how DSPy uses declarative signatures and automated optimizers to programmatically tune prompts, making LLM applications more robust and model-agnostic. The episode covers advanced techniques like using LLMs as judges for evaluation and building complex, multi-step workflows for production software.
Topics discussed
Introduction: Brett Kennedy and his new book on DSPy
Collaborating on the book and the origins of DSPy
What is DSPy? Automating prompt engineering
Signatures and the importance of typing in DSPy
The problem with manual prompt engineering
Optimizers: Automating prompt tuning and demonstrations
Hill climbing and optimizing for different LLMs
Robustness in production and the assembly code analogy
Future-proofing code and switching LLM providers
RealPython Course: Using Llama Index for RAG
DSPy benefits: Unified API and complex workflows
Building multi-step RAG workflows with DSPy
LLM as a Judge for consistent evaluation
Book structure: From intent classification to agents
Integrating tools and MCP servers with DSPy
Getting started with DSPy and surprising prompt results
Advanced optimizers and handling context windows
Where to find the book and tabular deep learning
Personal hobbies: Welding, guitar, and learning
Listen ad-free on Castria