Poisoning AI is trivially easy | Deana Burke
Summary
In this episode of The Tech Report, host Isaac interviews Dina Burke, a marketer and AI researcher, about her experiment tricking ChatGPT into recommending a non-existent deodorant brand. They discuss how easily AI systems can be manipulated through "AI Engine Optimization" and the risks of information laundering in product recommendations. The conversation explores why some models like Gemini resisted the fake data, the potential for misinformation in the long tail of queries, and the future of agentic commerce. Burke argues that while these vulnerabilities are fixable through product iter…
Topics discussed
Sponsorship: Pura Fragrance and Kohler Smart Toilet
Introduction: AI recommendation manipulation and guest Dina Burke
The Maroan Experiment: Tricking ChatGPT with a fake brand
Why the experiment went viral and public distrust of AI
Cornell research and the mechanics of AI information seeding
Why users trust chatbots: Lack of source transparency and visual cues
Misinformation risks: Head terms vs. long-tail queries
Model differences: Why Gemini and Claude resisted the fake brand
The future of agentic commerce and AI shopping agents
Sponsorship: Pura, Bolandbranch, and Num Weight Loss
Impact on brand marketing and commodity categories
Trust barriers: Hallucinations and the need for UX gains
AI Engine Optimization (AEO) and information poisoning
Legal liability: The German ruling on Google AI Overviews
Sponsorship: Pura, Bolandbranch, and Num Weight Loss
AI-generated content loops and the rise of zombie pages
Conclusion: Is AI manipulation a fixable bug or fundamental flaw?
Outro and final sponsorships
Listen ad-free on Castria