Uber’s AI infra, from the inside
Summary
Uday Mendicetti, a long-tenured Uber engineer, details the company’s evolution from a two-service monolith to a massive microservices ecosystem, highlighting internal tools like M3DB, Michelangelo, and Cadence. He explains how Uber’s pre-AI platform investments, such as DevPod and standardized IDLs, enabled rapid adoption of agentic AI. The episode covers new AI-specific infrastructure, including a Model Gateway for centralized LLM access, an MCP Gateway that auto-generates tool specs from existing APIs, and a Context Graph that maps relationships between people, code, and assets to improve…
Topics discussed
Introduction to Uber's internal tools and agentic AI
The challenge of agent permissions and the Worker airlock
Uday's tenure at Uber and early architecture (Dispatch/RTAPI)
Evolution from monolith to microservices and language standardization
Scaling engineering teams and global tech sites
Shift to developer platform and agentic AI focus
Foundational tools: YouOwn, M3DB, and observability
Michelangelo (ML platform) and Cadence (workflow engine)
Morpheus (experimentation) and build vs. buy strategy
Complex experimentation for two-sided marketplaces
Global traffic management, sharding, and failover
Monorepos and the transition to Mega Dev Pods
Standardizing agent context with top-level files
Current development landscape and tooling overview
Model Gateway: Abstracting LLM complexity
Integrating AI evals with Michelangelo infrastructure
Model Gateway evolution and client integration
Latency guardrails, attribution, and audit logging
AIFX wrapper and persona management across harnesses
Spend tracking and cost visibility for engineers
Responsible use tiers and managed agent limits
Real-time coaching and cost optimization feedback
MCP Gateway: Auto-generating tools from IDL specs
LLM-optimized descriptions and tool generation
CLI vs MCP interfaces and pre-AI platform foundations
Vendor-neutral building blocks and skills repository
Cross-harness plugin installation and configuration
Skill execution volume and repository management
Optimizing skills for token efficiency via CLI
Context Graph: Connecting siloed engineering data
Visualizing the Uber Context Graph
Impact of Context Graph on quality, cost, and latency
Interconnecting data sources and query history
Leveraging production graph infrastructure for Context Graph
Cortana AI assistant and 360-degree service health reports
Sponsor: TurboPuffer v3 storage architecture
Sponsor: Linear for human-agent coordination
Code review changes in the AI era
Upleveling lifecycle with AI foundations
The importance of planning for complex agent tasks
Minion: Uber's background coding agent
Minion v2 features and Mega Dev Pod integration
Increasing task complexity and Slack integration
DevPod flavors and Minion environment setup
Rapid DevPod creation for code changes
Optimizing CI/CD for the AI era
Self-healing CI and PR automations
Risk scoring and blast radius analysis for PRs
Maintaining human review for safety and incident cost
Sanity testing and end-to-end feature verification
Teaching agents to deploy Slate environments
Slate UI and agent-driven environment management
The critical role of Slate in microservice testing
Service mesh routing for reliable test isolation
Context capture for the Rider app screens
Capturing full request/response chains
Combinatorial testing and high-confidence validation
Replaying specific production requests on Slate
Evolution from Deputy/Cerberus to Slate routing
Agent-driven capture/replay and Edge Gateway
Rich context for debugging alerts and incidents
Shepard: AI-based large-scale code migrations
Fox: Self-improving codebases via continuous skills
Enrolling services into automated maintenance skills
Wisdom: Dogfooding and bug reproduction
Automated reproduction steps from user logs
Continuous improvement of debug assist and Minion
Ease of bug reproduction and custom tooling
Maximizing feature shipping and lifecycle rewiring
Defining engineering productivity in the AI era
Feature velocity vs. depth/velocity metrics
Measuring end-to-end cycle time with U-Plan
Changing definition of 'hands-on' engineering
Multitasking across AI sessions and verification
Automating customer support policy creation
Designing systems for automation vs. manual work
Closing remarks and recap
Capri session capture and CodeReview trends
Value of pre-AI developer platform teams
Listen ad-free on Castria