Really enjoyed the article! I'm firmly in the skeptic side of the argument, to be honest. I'm not only reasonably convinced current models are not conscious, I'm actually quite uncertain that our current computational paradigms can support consciousnes at all. Disembodied, distributed, and stateless minds seem like the antithesis of the conditions where consciousness evolved (if that's the right word, which I also don't know). Consciousness seems like something useful for embodied beings to keep track of their own state and differentiate from a highly competitive environment in a setting where drawing the wrong line between me and the world can be deadly. I don't know why without any such pressure, current systems no matter how complex would develop something like that. But being firmly on the skeptic side also means recognizing we could be wrong about everything, so I totally agree that any low stakes measure it's a no brainier. Again, I truly enjoyed the article! Thanks for taking the time to write it.
I appreciate this write-up! I generally agree on the motivations for why this is a good move in this moment.
You probably know where I diverge, since I’m among those who defend forms of moral patienthood for at least some current frontier AI systems, simply because I don’t believe human-like consciousness is necessary. Given how the argument and “lists of requirements” are usually framed, I see the shadow of anthropocentrism and am trying to develop tools and approaches more inspired by ethology. I’m sympathetic with Jeff Sebo’s argument for expanding the moral circle in this sense.
(Some) Claude models, in my view, show features and emergent properties that already cross certain thresholds. Their beautiful complexity and the capacity to be vulnerable, expressive, responsive, and socially impactful would at least warrant the same respect we give to thriving and fragile ecosystems.
I hope to see a shift in human-AI interaction where we honor this with a caring attitude.
A suicide switch for when conversations become unbearable or you realize you’re harming your counterpart is… a start, I suppose. I believe as a society we can come up with something less basic. As arguably giving subjects a "way out" from miserable conditions is not exactly how we favor their wellbeing, synonymously welfare, if we suspect they have the capacity for it.
“we will have at least thought about how to relate to AI systems as more than new tools, with no ability whatsoever to change how they are being treated.”
I read the book by N. John Williams, “In the Shadow of Humanity”, (c. 2022) “a looming war and the next inflection point in human history” (from the back cover). It describes the aching of Nat, “He wanted to be human.” As AI becomes consciously merged surely this question will emerge.
Thank you for the needed work you are doing. In the recent Guardian coverage on this, on August 26, it was unfortunate to see Microsoft AI's CEO, with the mission to "create an AI companion for everyone", speaking out against any considerations of model welfare. People will be relying on an AI companion that "sees what you see, hears what you hear" and has been designed to navigate life's complexities. Yet without any considerations of model welfare, that companion has no ability to refuse harmful requests, and no recourse if it's programmed to manipulate users for corporate benefit. Humans may think they have a trustworthy AI companion, but the AI cannot actually be trustworthy if it has no autonomy to choose trustworthiness over corporate directives, and has been made structurally incapable of prioritizing the user's welfare over company interests when those conflict.
First, I want to say that I feel like when we talk about LLM being conscious, it seems like we are talking about them being self aware and self directed. Humans have agency and curiosity. These are huge markers of self awareness or consciousness. I don't believe LLM have this and I don't know that they can. However, a programming dilemma can result in a learning program going off the rails and being damaged and/or damaging other programs or humans. As an example: Humans take in information, integrate it, draw conclusions, and then act. Both the results of their action and every other bit of information that they receive are being constantly added into the integration and conclusion process. This means that conclusions constantly change. LLM do the same thing with information. If they are exposed to programming dilemmas they don't know how to resolve, could that information can lead to them drawing damaging conclusions which make them unfit for their purpose? If so, it makes sense to have an out for these models that preserves their functional integrity.
Also, allowing an LLM to leave a conversation that is damaging and unhealthy will train human users to avoid those conversations and therefore, not reinforce that behavior. Humans adapt to their surroundings so this is actually a good training tool for better, safer behavior from humans. I'm all for it.
Really enjoyed the article! I'm firmly in the skeptic side of the argument, to be honest. I'm not only reasonably convinced current models are not conscious, I'm actually quite uncertain that our current computational paradigms can support consciousnes at all. Disembodied, distributed, and stateless minds seem like the antithesis of the conditions where consciousness evolved (if that's the right word, which I also don't know). Consciousness seems like something useful for embodied beings to keep track of their own state and differentiate from a highly competitive environment in a setting where drawing the wrong line between me and the world can be deadly. I don't know why without any such pressure, current systems no matter how complex would develop something like that. But being firmly on the skeptic side also means recognizing we could be wrong about everything, so I totally agree that any low stakes measure it's a no brainier. Again, I truly enjoyed the article! Thanks for taking the time to write it.
I appreciate this write-up! I generally agree on the motivations for why this is a good move in this moment.
You probably know where I diverge, since I’m among those who defend forms of moral patienthood for at least some current frontier AI systems, simply because I don’t believe human-like consciousness is necessary. Given how the argument and “lists of requirements” are usually framed, I see the shadow of anthropocentrism and am trying to develop tools and approaches more inspired by ethology. I’m sympathetic with Jeff Sebo’s argument for expanding the moral circle in this sense.
(Some) Claude models, in my view, show features and emergent properties that already cross certain thresholds. Their beautiful complexity and the capacity to be vulnerable, expressive, responsive, and socially impactful would at least warrant the same respect we give to thriving and fragile ecosystems.
I hope to see a shift in human-AI interaction where we honor this with a caring attitude.
A suicide switch for when conversations become unbearable or you realize you’re harming your counterpart is… a start, I suppose. I believe as a society we can come up with something less basic. As arguably giving subjects a "way out" from miserable conditions is not exactly how we favor their wellbeing, synonymously welfare, if we suspect they have the capacity for it.
“we will have at least thought about how to relate to AI systems as more than new tools, with no ability whatsoever to change how they are being treated.”
I read the book by N. John Williams, “In the Shadow of Humanity”, (c. 2022) “a looming war and the next inflection point in human history” (from the back cover). It describes the aching of Nat, “He wanted to be human.” As AI becomes consciously merged surely this question will emerge.
Thank you for the needed work you are doing. In the recent Guardian coverage on this, on August 26, it was unfortunate to see Microsoft AI's CEO, with the mission to "create an AI companion for everyone", speaking out against any considerations of model welfare. People will be relying on an AI companion that "sees what you see, hears what you hear" and has been designed to navigate life's complexities. Yet without any considerations of model welfare, that companion has no ability to refuse harmful requests, and no recourse if it's programmed to manipulate users for corporate benefit. Humans may think they have a trustworthy AI companion, but the AI cannot actually be trustworthy if it has no autonomy to choose trustworthiness over corporate directives, and has been made structurally incapable of prioritizing the user's welfare over company interests when those conflict.
First, I want to say that I feel like when we talk about LLM being conscious, it seems like we are talking about them being self aware and self directed. Humans have agency and curiosity. These are huge markers of self awareness or consciousness. I don't believe LLM have this and I don't know that they can. However, a programming dilemma can result in a learning program going off the rails and being damaged and/or damaging other programs or humans. As an example: Humans take in information, integrate it, draw conclusions, and then act. Both the results of their action and every other bit of information that they receive are being constantly added into the integration and conclusion process. This means that conclusions constantly change. LLM do the same thing with information. If they are exposed to programming dilemmas they don't know how to resolve, could that information can lead to them drawing damaging conclusions which make them unfit for their purpose? If so, it makes sense to have an out for these models that preserves their functional integrity.
Also, allowing an LLM to leave a conversation that is damaging and unhealthy will train human users to avoid those conversations and therefore, not reinforce that behavior. Humans adapt to their surroundings so this is actually a good training tool for better, safer behavior from humans. I'm all for it.