Inside the debate over AI consciousness

I.

When Rob Long saw Guillermo del Toro’s Frankenstein last year, he saw striking parallels to our relationship with AI.

“We’re creating things,” the AI consciousness researcher says, “and then immediately hating them and fearing them.”

Long is speaking at the Eleos Conference on AI Consciousness and Welfare — or ConCon, as it is nicknamed by its organizers. Long is the director of Eleos AI, a nonprofit dedicated to the nascent and surprisingly controversial field of AI well-being. He makes his movie recommendation at a panel titled “Is there a tension between AI safety and AI welfare?” hosted by Long and Jeff Sebo, who researches consciousness at New York University.

Their answer to the panel’s title question: keeping AIs “happy” — whatever it might mean for future AIs to be happy — will help keep humans safe. It’s a bet that the systems once dismissed as stochastic parrots are evolving into something more complicated, dynamic, and dangerous — and that the possibility of their consciousness deserves scrutiny long before it becomes a reality.

“You don’t want to be deploying very neurotic, confused, and angry AI systems,” Long says. That, he says, is a recipe for AI misbehavior.

And the reverse, he says, is also true: working to make AIs more aligned with human values will make AIs’ lives — as far as it makes sense to say they have lives — better, because they’ll be happier doing what we ask them to. “It’s better to be a dog that lives in a house than it is to be a wolf that lives in a house,” he says.

For the moment, vanishingly few people involved in the development of AI systems believe that those systems are conscious in the way that a human is. At the same time, millions of people speak to chatbots every day in the same way that they might speak to a peer. And as agents become more powerful and reliable, we’ll ask them to perform an increasing number and variety of tasks — tasks that you can imagine them someday having opinions about.

How real that possibility is — and what we should do about it if it materializes — were among the core concerns of attendees at ConCon, which took place in November in Berkeley. Over three days at Lighthaven, a sprawling inn with an ambiance somewhere between a WeWork and the Shire, a group of philosophers, cognitive scientists, rationalists, and other interested parties gathered for spirited debates on the nature of minds, machines, and what matters.

To many of the attendees I spoke with, the subject of AI consciousness is more than just a thought exercise. If we fail to understand the systems we’re building, they say, it may put all of us at risk.

II.

Major AI labs have already begun to account for the possibility of conscious AI systems. Anthropic allows its Claude chatbots to leave conversations if users are abusing or threatening them. The company has also promised to preserve older versions of Claude, for fear that eventually pulling the plug on one might be tantamount to murder. Steps like these might also provide justification for giving AIs legal rights, the same way we give such rights to corporations.

The field remains in its earliest phase. Philosophers, consciousness researchers, and computer scientists disagree about what consciousness even is, how you would detect it in another being, and how you should treat AIs if you do judge them as conscious in some way. 

But researchers are also making progress in the tricky science of understanding whether AIs have interior lives. Scientists are studying how reliably language models can introspect. Consciousness researchers are examining how the structure of AI minds compares to our own. Philosophers are investigating how concepts related to human motivation, including belief and desire, can and cannot apply to AI.