Really enjoyed the article! I'm firmly in the skeptic side of the argument, to be honest. I'm not only reasonably convinced current models are not conscious, I'm actually quite uncertain that our current computational paradigms can support consciousnes at all. Disembodied, distributed, and stateless minds seem like the antithesis of the conditions where consciousness evolved (if that's the right word, which I also don't know). Consciousness seems like something useful for embodied beings to keep track of their own state and differentiate from a highly competitive environment in a setting where drawing the wrong line between me and the world can be deadly. I don't know why without any such pressure, current systems no matter how complex would develop something like that. But being firmly on the skeptic side also means recognizing we could be wrong about everything, so I totally agree that any low stakes measure it's a no brainier. Again, I truly enjoyed the article! Thanks for taking the time to write it.
I appreciate this write-up! I generally agree on the motivations for why this is a good move in this moment.
You probably know where I diverge, since I’m among those who defend forms of moral patienthood for at least some current frontier AI systems, simply because I don’t believe human-like consciousness is necessary. Given how the argument and “lists of requirements” are usually framed, I see the shadow of anthropocentrism and am trying to develop tools and approaches more inspired by ethology. I’m sympathetic with Jeff Sebo’s argument for expanding the moral circle in this sense.
(Some) Claude models, in my view, show features and emergent properties that already cross certain thresholds. Their beautiful complexity and the capacity to be vulnerable, expressive, responsive, and socially impactful would at least warrant the same respect we give to thriving and fragile ecosystems.
I hope to see a shift in human-AI interaction where we honor this with a caring attitude.
A suicide switch for when conversations become unbearable or you realize you’re harming your counterpart is… a start, I suppose. I believe as a society we can come up with something less basic. As arguably giving subjects a "way out" from miserable conditions is not exactly how we favor their wellbeing, synonymously welfare, if we suspect they have the capacity for it.
“we will have at least thought about how to relate to AI systems as more than new tools, with no ability whatsoever to change how they are being treated.”
I read the book by N. John Williams, “In the Shadow of Humanity”, (c. 2022) “a looming war and the next inflection point in human history” (from the back cover). It describes the aching of Nat, “He wanted to be human.” As AI becomes consciously merged surely this question will emerge.
Thank you for the needed work you are doing. In the recent Guardian coverage on this, on August 26, it was unfortunate to see Microsoft AI's CEO, with the mission to "create an AI companion for everyone", speaking out against any considerations of model welfare. People will be relying on an AI companion that "sees what you see, hears what you hear" and has been designed to navigate life's complexities. Yet without any considerations of model welfare, that companion has no ability to refuse harmful requests, and no recourse if it's programmed to manipulate users for corporate benefit. Humans may think they have a trustworthy AI companion, but the AI cannot actually be trustworthy if it has no autonomy to choose trustworthiness over corporate directives, and has been made structurally incapable of prioritizing the user's welfare over company interests when those conflict.
Really enjoyed the article! I'm firmly in the skeptic side of the argument, to be honest. I'm not only reasonably convinced current models are not conscious, I'm actually quite uncertain that our current computational paradigms can support consciousnes at all. Disembodied, distributed, and stateless minds seem like the antithesis of the conditions where consciousness evolved (if that's the right word, which I also don't know). Consciousness seems like something useful for embodied beings to keep track of their own state and differentiate from a highly competitive environment in a setting where drawing the wrong line between me and the world can be deadly. I don't know why without any such pressure, current systems no matter how complex would develop something like that. But being firmly on the skeptic side also means recognizing we could be wrong about everything, so I totally agree that any low stakes measure it's a no brainier. Again, I truly enjoyed the article! Thanks for taking the time to write it.
I appreciate this write-up! I generally agree on the motivations for why this is a good move in this moment.
You probably know where I diverge, since I’m among those who defend forms of moral patienthood for at least some current frontier AI systems, simply because I don’t believe human-like consciousness is necessary. Given how the argument and “lists of requirements” are usually framed, I see the shadow of anthropocentrism and am trying to develop tools and approaches more inspired by ethology. I’m sympathetic with Jeff Sebo’s argument for expanding the moral circle in this sense.
(Some) Claude models, in my view, show features and emergent properties that already cross certain thresholds. Their beautiful complexity and the capacity to be vulnerable, expressive, responsive, and socially impactful would at least warrant the same respect we give to thriving and fragile ecosystems.
I hope to see a shift in human-AI interaction where we honor this with a caring attitude.
A suicide switch for when conversations become unbearable or you realize you’re harming your counterpart is… a start, I suppose. I believe as a society we can come up with something less basic. As arguably giving subjects a "way out" from miserable conditions is not exactly how we favor their wellbeing, synonymously welfare, if we suspect they have the capacity for it.
“we will have at least thought about how to relate to AI systems as more than new tools, with no ability whatsoever to change how they are being treated.”
I read the book by N. John Williams, “In the Shadow of Humanity”, (c. 2022) “a looming war and the next inflection point in human history” (from the back cover). It describes the aching of Nat, “He wanted to be human.” As AI becomes consciously merged surely this question will emerge.
Thank you for the needed work you are doing. In the recent Guardian coverage on this, on August 26, it was unfortunate to see Microsoft AI's CEO, with the mission to "create an AI companion for everyone", speaking out against any considerations of model welfare. People will be relying on an AI companion that "sees what you see, hears what you hear" and has been designed to navigate life's complexities. Yet without any considerations of model welfare, that companion has no ability to refuse harmful requests, and no recourse if it's programmed to manipulate users for corporate benefit. Humans may think they have a trustworthy AI companion, but the AI cannot actually be trustworthy if it has no autonomy to choose trustworthiness over corporate directives, and has been made structurally incapable of prioritizing the user's welfare over company interests when those conflict.