AI User Group #19: Leo Gets (DGX) Sparky
Summary
The AI User Group panel discusses local AI hardware setups, comparing NVIDIA DGX Spark, Mac M-series, and older GPUs like the RTX 3090. Host Leo Laporte shares his experience running DeepSeek and Qwen models locally, highlighting performance benchmarks and energy costs. The group also covers agent orchestration tools like Hermes and Pi, debating the efficiency of local compute versus frontier models for coding tasks.
Topics discussed
Intro: Nerdy Drunk and guest introductions
Local AI setup: RTX 3090 vs Mac and GLM models
Benchmarking: Token speeds on NVIDIA vs Apple Silicon
Hermes harness, Pi interface, and vision model tests
Hardware debate: $5k budget for Spark vs Mac Studio
Used GPU market: Buying RTX 1090s on eBay
Running costs: Electricity prices and power consumption
Model comparisons: Fable, Kimi, and DeepSeek performance
Coding agents: Graft, code graphs, and token efficiency
Natural language programming and future of code
AI history: From spicy autocorrect to coding agents
Defining AI Harnesses: The body for the LLM brain
Neuroscience: Human brains vs Von Neumann architecture
AI addiction and integrating agents into daily life
AI Watermarking: Techniques and privacy concerns
Authenticity debate: Vermeer, Tim's Vermeer, and AI art
Outro: Show schedule, newsletter, and sign-off
Listen ad-free on Castria