Common mistakes about AI consciousness
Must do better
The question of whether AI systems could be conscious—i.e., have subjective experiences—is an issue that we will have to wrestle with a lot more in coming years. Here are some common mistakes that people make when discussing AI consciousness.
(1) Taking behavior at face value
Sometimes people claim that they can just tell whether an AI system is or is not conscious, simply by interacting with it. But this is dubious.
To see why it’s dubious with AI, consider why it is not especially dubious with other humans. If I see you jerk your hand back and say “ouch!” after touching the stove, it’s not particularly problematic for me to infer that you are experiencing pain. This is a reasonable and reliable inference because, in the space of possible minds, you are very similar to me, and I know that when I behave that way, this behavior is usually accompanied by an experience of pain. It stands to reason (absent any undermining evidence, like learning that you were joking or have congenital analgesia) that this relationship holds for you as well.
This relationship does not hold in the case of AI. In the space of possible minds, AI systems can be very different from us—even more different than the most exotic and strange animals. This difference can break the behavior-experience relationship. As I wrote after the bizarre Bing Chat release:
The putative self-reports of large language models like Bing Chat are an unreliable and confusing guide to sentience, given how radically different the architecture and behavior of today’s systems are from our own. So too are whatever gut impressions we get from interacting with them. AI sentience is not, at this point, a case of “you know it when you see it”.
Relying too much on behavior could be wildly misleading for determining AI consciousness - our psychological biases could lead us to both over-attribute and under-attribute consciousness, depending on the “surface” features of AI systems, including clearly irrelevant ones like whether they have a cute avatar.
(2) Overconfidence
There is no consensus scientific theory of consciousness, and there are a lot of things we still don’t know. And consciousness is a philosophical and conceptual thicket. As a result, I think that extremely confident claims about AI consciousness are often a mistake. Some claims that I think are way too overconfident given our current knowledge include:
It’s been proven that no computer system could possibly be conscious.
It’s been proven that AI could, in principle, be conscious.
There’s no way we’ll have conscious AI anytime soon.
We’ll definitely have conscious AI soon (or already).
Consciousness is obviously an illusion and/or a non-issue.
[insert favorite] scientific theory of consciousness is true.
(3) Total agnosticism
At the same time, difficulties about consciousness don’t mean that it’s impossible to say anything at all about AI consciousness. One error is to just throw up one’s hands and declare the whole issue to be impossible. Matters get a lot better once you stop demanding certainty and talk in terms of probabilities and evidence. And we do know some things about consciousness - most centrally, we know something about the brain regions and processes that are associated with it in humans. With this knowledge, we can tentatively make claims about which AI systems might be more likely and less likely to be conscious, depending on how closely their processes resemble the ones we know about from consciousness science. This is the approach and way of speaking that we adopt in “Consciousness in AI: Insights from the Science of Consciousness”.
(4) Considering only one (or zero!) AI systems
Much of the recent interest in AI consciousness has understandably focused on large language models. But these systems are not even necessarily the best candidates for consciousness that exist today. For one thing, many of the proposed necessary conditions that LLMs allegedly lack—such as embodiment and agency—are much more plausibly satisfied in other kinds of systems. For example, robots are much more straightforwardly agentic and embodied than GPT-4 is.
If we want to know whether we could soon have conscious AI systems, we can’t focus on only one of the myriad existing AI systems that exist. If it is possible at all for artificial systems to be conscious, then asking simply “Is AI conscious?” is rather like asking, “Are organisms conscious?” The answer to that question is presumably: well, it depends on which organism. The same is true in AI.
(5) Conflating consciousness and cognitive sophistication (or understanding, intelligence, rationality, etc.)
“Consciousness” is not the same thing as being smart—and certainly not the same thing as having human-level or human-like cognitive capabilities. Many animals are plausibly conscious while falling far “short” of human cognitive sophistication: bees, chickens, dogs. By the same token, it is an open possibility that there could be AI systems that are conscious while falling short of human-level cognitive capabilities. So conscious AI should not be confused with “AGI”, human-level AI, superintelligence, or anything in that vicinity. And arguments that AI could be conscious should certainly not be confused with arguments about AI risk: such arguments are often explicitly neutral on whether a dangerous AI would be conscious, and do not rely on this assumption for their success.
Have a great rest of the week, readers! And steer clear of these errors 🫡