AI Computing Hardware - Past, Present, and Future
Summary
This episode explores the critical intersection of AI and hardware, tracing the evolution from early custom machines to modern GPU-driven data centers. Hosts Jeremy and Frank discuss how scaling laws and the "bitter lesson" shifted focus toward massive compute power, highlighting the "memory wall" challenge where logic outpaces memory latency. They examine the hierarchical structure of data centers, detailing how memory types like HBM and SRAM support the exponential growth of AI infrastructure.
Topics discussed
Sponsor intro and episode overview
Host introductions and hardware focus
History of AI hardware from Turing to Deep Blue
Rise of GPUs, scaling laws, and the Bitter Lesson
Moore's Law, memory bottlenecks, and compute limits
Memory hierarchy, HBM, and inference vs training
Parallelism types and GPU architecture basics
NVIDIA GB200 system and data center infrastructure
Chip packaging, HBM, and semiconductor fabrication
Process nodes, TSMC, and fab economics
EUV photolithography and ASML technology
Export controls, supply chain, and conclusion
Listen ad-free on Castria