Why Medical AI Needs a Referee | Protege's Engy Ziedan
Summary
In this episode, Daisy Wolf and Eva Steinman interview Ng Ziden, co-founder of Protege, about the critical need for independent AI evaluations in healthcare. They discuss how current benchmarks fail to capture real-world clinical performance and subtle misalignments, such as bias in insurance denials or ambient scribing. Ziden argues that an independent "referee" is essential to continuously test models in live settings, ensuring safety and trust as AI agents become integrated into high-stakes medical workflows.
Topics discussed
Introduction: The healthcare AI measurement problem
Guest introduction: Ng Ziden of Protege
Protege's origin and the value of real-world data
Evolution of customers from startups to foundation models
Economic importance of AI evaluations and pricing
Information asymmetry in healthcare and AI trust
Catastrophic failures vs subtle bias and misalignment
The lack of independent oversight for clinical AI
Controversy over general vs vertical AI performance
Challenges of static benchmarks in a dynamic field
The need for continuous, real-time monitoring
Why acing exams does not equal clinical competence
Protege's role as an independent arbiter and referee
Protege's competitive advantage and impartiality
Why government regulation is too slow for AI
Preventing data contamination in benchmarks
AI credentials vs human medical certification
Closing thoughts on self-regulation and trust
Outro and podcast disclosures
Listen ad-free on Castria