Seven questions for Kathleen Finlinson
Kathleen Finlinson is head of strategy at Eleos AI Research, a brand new research organization working on AI sentience and welfare. Kathleen and I recently launched the org together, and published a big report on AI welfare.
At the tail end of a long work week, we chatted and I asked her some questions to help readers get to know Kathleen—you’ll definitely be seeing a lot more of her. Thanks to Kathleen for an excellent conversation!
1. What motivated you to work on AI welfare with Eleos?
Kathleen: I found it very compelling to work with you, because we were already working together. And I knew that you’re an exceptionally great colleague, and that we were enjoying working together.
Rob: Okay I’m going to have to immediately interject with a note to readers that I didn’t know you were going to say that. So this doesn’t look like a propaganda piece.
Kathleen: (laughs) Fair enough. Also, I felt that we were in this position that is rare when you're founding an organization, which was that we have a bunch of people already supporting us and a bunch of people wanting to hear what we have to say. So it created this very appealing role for me to step into.
But then also, of course, I am interested in AI sentience and welfare. I had been coming to your digital minds philosophy reading group for several months. I always found it to be a very interesting discussion full of smart people who I enjoyed working with. I'm not a philosopher by background, but I’ve had a side interest in philosophy of mind for many years.
As far as AI consciousness and welfare goes, I think it's just a fascinating field, and potentially very important. I have had a passion for AI safety for a long time, and for forecasting how the development of AI is going to shape the future. And it seemed clear that AI consciousness and welfare has some kind of interesting role to play in the mix of that field, and didn't have very many people working in it.
2. Do you have any theories about why you were psychologically pre-disposed to take AI welfare seriously?
Kathleen: Well, doesn't everyone find it interesting and emotionally compelling?
Rob: (laughs) No!
Kathleen: Oh, that's interesting. I thought I was just totally normal in this way.
Rob: I guess you're better at not being on Twitter than me, but one thing you'll notice is that on tweets about our paper it’s like, about 20% of the replies are people being like “this is the stupidest thing I've ever heard of, what is wrong with you”.
Kathleen: (laughs) Oh right. Well, I guess I have spent enough time with Carl Shulman. And given my interest in AI forecasting, I had already spent some time thinking about digital minds, as many futurists do.
I think I also have a history of extending moral concern to a broader class of beings than is maybe typical. I've done some work in animal welfare - some of my work has been in bird welfare. A lot of people mainly care about mammals. I also think that insect welfare and fish welfare are very important and underrated. To me, I think it's just very natural to think that AI systems, or various forms of digital minds, could be conscious and morally relevant in all the relevant senses.
3. What reactions do you get when you talk about your new job?
Kathleen: It's probably just the circles I hang out in, but mostly everyone just thinks it's super fascinating. Or I guess it’s not just my circles, because that includes people like Uber drivers or people I meet at the store. It seems like, pretty uniformly, people think it's just one of the most interesting topics out there.
Also, I’ve noticed everyone I meet believes they have a really valuable take to share.
Rob: (laughs) Well, the thing about consciousness is, everyone's an expert.
Kathleen: I also think it's very common for people to conflate a bunch of different things that I personally believe are conceptually distinct, such as consciousness, moral patienthood, intelligence. When I say 'consciousness,' people often basically think I mean intelligence, because they start to talk about “Oh yeah, if AIs could know all kinds of things, or have the capability to be able to take over, then…"
4. A lot of your previous work has been on forecasting AI progress. How has that influenced the way you approach AI welfare?
Kathleen: I am very trained to think about, not just what AI systems are like now, but what they are going to be like in five years, 10 years, 20 years. Obviously there's always a lot of uncertainty, but I've spent a lot of my time analyzing the case that AI will continue to develop, and that the size of models, the amount of compute, that data going into training them, will continue to scale rapidly.
Which obviously is something that sounds speculative. But back when I started studying this in 2018 it sounded very speculative—and then it started to come true.
5. You’ve spent significant time exploring meditation, including living in a Zen monastery for 6 months. Have those experiences shaped your perspective on consciousness?
Kathleen: The Zen tradition is known for questioning what parts of our experience we identify with. I can't say I have deeply understood and experienced Zen (or other Buddhist) teachings on this, but it's shifted my beliefs and experience—I think it’s good to have spent time seeing how intuitive self-models can break down.
For example, I was on a retreat once, and I remember looking at the carpet, and at my legs on the carpet—and seeing that in some sense, both of them were completely out of my control. Sometimes some part of me just decides to move my legs. But there's some other part of me that is just observing that, and in some sense I'm really just observing everything, both externally and internally.
Again, experiences like this don’t have any obvious or straightforward connection to AI consciousness, but I think it’s really helpful to have viscerally experienced how a lot of our commonsense understanding of self and consciousness can just be wrong.
6. What questions are you most excited to explore in your role at Eleos?
Kathleen: LLMs are not the only AI systems that we might want to study at Eleos, but they are a big component. And one interesting thing about thinking about the welfare of LLMs is that you can talk to them and they'll say things. I think it's a really interesting question: when you input text to an LLM and it outputs text, how meaningful is the answer that it gives you, if you're asking it questions about itself? And would it even mean for the answer to be meaningful? I think the work by some of our close collaborators on introspection is extremely important, and we need to keep promoting that line of research. There’s a really interesting mix of conceptual and empirical questions there.
And as head of strategy, I am really focused on what effects are we trying to have in the world. We obviously want a future in which both biological and digital minds are flourishing, but we need to get more specific than that. Right now, the primary source of power and influence in the world is still biological minds, primarily human—although in some cases, for example my home, ruled by a feline mind. As an AI forecasting researcher, I thought a lot about what more powerful AI systems might look like. So I am just very interested in questions like: how can our work at Eleos affect the ways that future interactions between humans and powerful AI systems play out in the most helpful ways?
7. Any music and movie recommendations for readers?
Rob: What album have you listened to most in your life? What’s your pitch for it?
Kathleen: Górecki's Third Symphony. It's very sad, and very beautiful. I'm a huge fan of modern classical music, which, to most people, is very inaccessible. But this is an example of modern classical music that is definitely on the accessible end, and it's kind of a bit minimalist. It's very emotional, and it just contains these very beautiful messages of hope in times of enormous darkness.
Rob: Favorite film?
Kathleen: I'm really into Ghibli films. I can’t pick just one. I’ll say: Ponyo, Totoro, and Princess Mononoke.
Rob: I guess Ghibli is less contrarian than modern classical. There are lots of Ghibli pitches—but what’s the Kathleen Finlinson pitch? How sell your friends on it?
Kathleen: I would say that one of the greatest storytellers of our time just happens to create art in this Japanese anime format, that on the surface level is for children, but is in fact very deep and appropriate for adults—while still being, at the same time, often lighthearted.
Rob: Great—thanks for chatting!
Kathleen: My pleasure! Good evening Robert Long.
Rob: Good evening Kathleen Finlinson.
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with only primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461, and here is a video of Jeff Krichmar talking about some of the Darwin automata, https://www.youtube.com/watch?v=J7Uh9phc1Ow