Uncharted waters

Consciousness in animals and AIs

One of the most common questions I get when I say that I work on AI consciousness¹ is, “where do you even start?”²

My answer usually points to the study of animal consciousness, the scientific field that I believe is the best starting off point for thinking about AI consciousness. Scientists who think about (e.g.) octopus consciousness already grapple with the many puzzles that present themselves whenever we try to take our understanding of consciousness, already so tenuous, and extend it beyond the human case. 

Both animal consciousness and AI consciousness force us to assess consciousness under challenging circumstances. It’s hard enough with humans. But we do actually have a pretty good idea of when other (adult, awake) humans are conscious—clever-sounding questions like “how you do know I’m conscious?” notwithstanding. In a previous post, I wrote about the links between experience and behavior that we can count on in the human case:

If I see you jerk your hand back and say “ouch!” after touching the stove, it’s not particularly problematic for me to infer that you are experiencing pain. This is a reasonable and reliable inference because, in the space of possible minds, you are very similar to me, and I know that when I behave that way, this behavior is usually accompanied by an experience of pain. It stands to reason (absent any undermining evidence, like learning that you were joking or have congenital analgesia) that this relationship holds for you as well.

But these links are weaker with animals and even moreso with AI.

Weakening links

With animals—octopuses, let’s say—we start running into problems. We will have these problems, and more, with AI systems.

Unlike humans, animals:

• Cannot communicate with us about what they are experiencing. Humans can tell you “I am experiencing pain”; octopuses are notoriously taciturn. 

• Have different brain structures (or computational structures) than we do. Humans all work basically the same way; octopuses have so many neurons in their arms that it has made people wonder if they might have a distributed, dis-unified experience of the world.³

• Can display behaviors that “mean” something different than they would for us. If I stretch myself out, that might mean that I want to take a nap; if an octopus stretches itself out, this might indicate that it feels threatened and is trying to appear larger. 

Still, animal consciousness does have some helpful features. Most importantly, animals are related to us. This gives us a few footholds—but these footholds get weaker the less closely animals are related to us, and get even weaker when we get to AI.

1. Brains. In spite of our differences, many animals, especially vertebrates, have brain structures that are analogous to structures associated with consciousness in humans. This allows us to draw some parallels between animal and human neural activity.

The further we get from humans on the tree of life, the harder this becomes. In an Asterisk piece on animal and AI intelligence, I noted some examples of how brain differences have made us misunderstand animals:

• Bird intelligence was overlooked for a long time in part because birds do not have a neocortex (very very roughly, the “smart” part of our brain). But it’s now hypothesized that birds use alternative brain regions to perform the same function as the cortex. 

• Bee intelligence was overlooked for a long time because they have tiny brains. Karl von Frisch, who discovered complex bee behavior like the waggle dance, nonetheless remarked, “The brain of a bee is the size of a grass seed and is not made for thinking.” But we now know that bees pack their neurons more densely than we do, which allows them to fit quite a lot of thinking into that grass seed.

AI systems extend these differences even further. Their architectures can look, at least at first glance, quite different from the human brain. To be sure, there may be broad similarities at a low level—they are “neural” networks after all. But AI systems don’t need to have anything like the same brain structures as humans. At the very least, there’s no clean mapping.

2. Life-world. Animals live in the “same world” that we do. Animals move around in a three-dimensional world of mid-sized objects—rocks, plants, other animals—using sensory systems that are broadly similar to our own. This shared sensory experience gives us a rough intuitive grasp on how they might perceive and be conscious of their environment.

But this already breaks down in the animal kingdom. For example, bats have echolocation, and it’s hard if not impossible for us to comprehend what it would be like to experience the world via echolocation. What is it like to be a bat?

Turning to AI—if we knew that large language models were conscious, how would we comprehend what they are conscious of? Their sensory input would be something like pure text. What is it like to be ChatGPT?

3. Needs. Animals have similar basic needs and drives as humans: maintaining homeostasis, seeking food, and avoiding danger. These shared needs give us a foothold in understanding their behaviors and potential experiences.

But what does ChatGPT need? If it were conscious, what would its desires be? This is much harder to get a handle on.

Gaming
Moreover, studying AI systems often presents a problem that doesn't exist in animal studies: what Kristin Andrews and Jonathan Birch call the gaming problem. Animals (usually) have no incentive to mimic our behavior; whereas AI systems are often explicitly designed to produce human-like outputs. This creates a fundamental challenge in interpreting AI behavior. Social chatbots, for example, are explicitly optimized to seem emotional and conscious.

Animals, as strange as they are, will still always have some degree of biological relatedness to us. AI systems might be even further from us. That’s not to say that it would be impossible for an AI system to have human-like experiences; it’s just possible for them to have a much broader range than animals, realized in a much broader range of ways.

Looking forward

Given these challenges, we need to develop new approaches to assessing AI consciousness that are resistant to gaming and go beyond potentially misleading surface-level behavior. Rather than relying solely on behavioral markers, we should also try to understand the computational processes that underlie these behaviors.

However, this requires a more comprehensive understanding of consciousness itself. And studying animal consciousness might help with this too. As Andrews and Birch argue, investigating independently evolved instances of consciousness can help us distinguish between essential features of consciousness and those that are merely contingent to human experience. These insights can guide the development of more general theories of consciousness.

Getting a better handle on these questions is not merely an academic problem; there are, I have argued risks associated with either over- or under-attributing consciousness to AI systems. Amidst these risks and the pace of AI progress, let’s hope that we can find ways to make progress on these difficult questions fast.

Thank you to Rosie Campbell, Steve Newman, and Julius Simonelli for conversation and feedback.

1. By “consciousness”, I mean what philosophers call “phenomenal consciousness”—roughly, having subjective experiences like seeing the color red, feeling pain in a stubbed toe, or getting angry. Or, as we will see, potentially having experiences that are not accessible to humans, like what bats might experience via echolocation.

2. Others include “Who did you say you know here?” and “Do you know where the bathroom is?”

3. Carls-Diamante (2017): “If the octopus were conscious, what would its consciousness be like? This paper investigates the structure octopus consciousness, if existent, is likely to exhibit. Presupposing that the configuration of an organism’s consciousness is correlated with that of its nervous system, it is unlikely that the structure of the sort of conscious experience that would arise from the highly decentralized octopus nervous system would bear much resemblance to those of vertebrates. In particular, octopus consciousness may not exhibit unity, which has long been assumed to be the normal or default structure of consciousness.”