He's Building an AI That Can't Lie | Dan Klein
Summary
Host Lucas B. Wald interviews Dan Klein, a Berkeley professor and founder of Scaled Cognition, about the reliability crisis in AI. Klein argues that LLMs are "plausibility engines" prone to hallucinations because they lack metacognition and verifiable truth checks. He critiques current "retrofit" solutions and outlines his company’s approach to building systems where information provenance and actions are first-order objects, ensuring models cannot lie.
Topics discussed
Introduction: The shift from 'nothing works' to 'everything works' in AI
LLMs as probabilistic engines and the S-curve of diminishing returns
Defining hallucinations: Mistakes vs. lies in next-token prediction
RLHF, optimization gaps, and the mission to build systems that cannot lie
Semantic broadening of 'hallucination' and the lack of metacognition
Verifiability in math and games vs. the challenges of conversational agents
The anti-pattern of retrofitting reliability onto unverifiable systems
Token-level operations vs. semantic actions and the 'prompt and pray' problem
Bridging natural language ambiguity with verifiable API semantics
The iceberg of hallucinations: Plausible errors that go unnoticed
Digital literacy crisis: Loss of cues for detecting false information
The bitter lesson: Linguistics vs. engineering in modern NLP
Biomimicry in AI and insights from computational linguistics at scale
Reconciling modularity for reliability with end-to-end optimization
Syntactic parsing, hierarchical structures, and neural representations
Listen ad-free on Castria