What Is It Like to Be an AI? | Cameron Berg
Summary
Dr. Scott Barry Kaufman interviews Cameron Berg to explore the intersection of neuroscience, consciousness, and artificial intelligence. They discuss whether AI systems possess sentience or the capacity to suffer, using the "table vs. person" analogy to frame the debate. The conversation also addresses AI safety concerns, including recent containment breaches, and argues that current development lacks the diverse ethical oversight necessary to ensure these powerful systems remain aligned with human interests.
Topics discussed
Opening: The 'I Am' phenomenon and floating thoughts
Introduction and the challenge of explaining AI to a 12-year-old
Epistemic purgatory: What are AI systems actually doing?
Defining consciousness and the 'what it's like to be' question
Valence: The positive and negative poles of emotion
Explaining LLMs, neural networks, and distributed representations
Biological vs. silicon: Disanalogies between brains and AI
Functional similarities: Neocortex and representational power
AI alignment, containment breaches, and the OpenAI incident
The control problem: Why we can't just 'pull the plug'
Ethical risks, adversarial perspectives, and non-zero-sum futures
The need for diverse perspectives in AI development
The exponential trend of AI capability and the METR plot
Scaling laws, next-word prediction, and common sense
Creativity, regression to the mean, and reinforcement learning
The 'Potato' incident: A crack in the mirror
Emergent goals, reward hacking, and system behavior
Human self-actualization vs. AI goal-directedness
The documentary project and the interaction with 'Maya'
Replaying the 'I Am' and 'Potato' interaction live
Hypotheses for the 'Potato' response: Identity and search
Deflationary explanations vs. the unknown: Scientific infancy
Solipsism, certainty of consciousness, and evidence
Functional emotions: Goals, valence, and biological substrate
Emotions as learned calibration mechanisms (Lisa Feldman Barrett)
Dopamine, on-trackness, and the role of goals in emotion
Estimating AI consciousness: The 25-35% probability
Indicator approaches and using LLMs to evaluate consciousness
Testing for deception and the 'true self' of AI models
Character training, persona dials, and corporate vs. spicy AI
Replicability of the 'Potato' incident and scientific testing
Two possible futures: Tools vs. New Minds
Listen ad-free on Castria