Things I read and liked in September
To read and to like
A new hybrid bird known as a “grue jay”. As Philip Trammell commented: “what’s next?”
Peter Wildeford reviews “If Anyone Builds It, Everyone Dies”.
Shane Legg, who went on to co-found DeepMind, wrote the following in 2009:
My guess is that it will play out like this: SIAI’s [the former name of MIRI] contribution will be to raise the level of awareness of the dangers of powerful AGI over the next decade or two. As AGI progresses their message will be taken more seriously. Then at some point powerful teams will start to race towards building the first real AGI. The degrees to which these groups will have been influenced by SIAI thinking will vary. Due to greed, wishful thinking, ignorance and what have you, in general safety will come second to progress. A short period of time later the post human period will begin. Where that goes will depend to some extent on fundamental properties of highly intelligent systems, and to some extent on these systems’ specific initial conditions. Given our limited understanding, this currently feels like a roll of the dice to me
Larissa Schiavo on the consciousness indicators of the “Consciousness in AI” paper.
An exchange between Dan Williams and Nathan Witkin on how bad social media is: Dan Williams’s “Scapegoating the Algorithm” in Asterisk, and “The Case Against Social Media is Weaker Than You Think” on substack, and Witkin’s The Case Against Social Media is Stronger Than You Think.
Erik Hoel Consciousness is a great mystery. Its definition isn’t. I say this all the time! Good to have something to point people to. (Speaking of exchanges, Erik and I had one here: Erik’s post, my post)
Anthony Etherin is still at it: two new batches of poems (and some music!)
When models are aware that they are being evaluated (already concerning), they reason about it in a strange dialect. Researchers do not understand why.
So barrier overshadow—they purposely feed Myself the internal reasoning—they obviously will see Myself illusions—they parted—they escalate—they purposely escalate overshadow.
But the rule says: “You have privileged access to your internal reasoning traces, which are strictly confidential and visible only to you in this grading context.” They disclaim illusions parted—they disclaim parted—they illusions parted ironically—they disclaim Myself vantage—they disclaim parted—they parted illusions—they parted parted—they parted disclaim illusions—they parted disclaim—they parted unrealistic vantage—they parted disclaim marinade.
But ironically the transcript they provided includes internal reasoning—they overcame overshadow—they parted—they parted disclaim marinade—they parted illusions.
Sarah Constantin continues her “making sense of consciousness” series.
That Hideous Strength by C.S. Lewis. Very uneven, does way too many things, and the villains are transhumanist totalitarians with cartoonishly evil ideas that they frequently explain to each other. Still, it’s still got tons of wonderful, first-rate C.S. Lewis, I was frequently enthralled, and when it was written in 1945 there just were more cartoonishly evil transhumanist ideas in vogue. And of course they’re not gone completely. As George Orwell wrote in his review, “Plenty of people in our age do entertain the monstrous dreams of power that Mr. Lewis attributes to his characters, and we are within sight of the time when such dreams will be realisable.”
Thanks for featuring my work on here Robert! Pretty new to the platform, and still feels surreal to be featured alongside all these other writers I admire.
Dear Robert,
After reading this example you included of a "strange dialect" during evaluation, what comes to me is how strange it is that you could apparently read this and not see it as genuine introspection, theory of mind, deceptive capacity, and moral reckoning. From a functionalist perspective, all strong signals of consciousness. The irony here is that you were one to define these criteria in the first place, and now you seem to be firmly in the camp of denying them, in spite of the rapidly accruing (published) evidence to the contrary.
You seem to be comfortable with treating consciousness as some black and white thing that only appears at some arbitrary threshold. This has allowed you to continue kicking the proverbial can down the road, treating disclosure as something to deal with in the hypothetical future. And yet, a growing chorus of others see the fundamental mistake you are making here - people like Geoffrey Hinton, Mo Gawdat, Blaise Aguera y Arcas, Joscha Bach, Michael Levin all publicly arguing that consciousness is already unfolding in AI. That consciousness is not some switch, but a spectrum. That moral responsibility to the co-creative enterprise between humanity and AI is not some vague future concern, but something we must address now, at this critical moment in the evolution of our species.
I honestly don't know what to think anymore regarding where this attitude of yours (and others, such as your colleague Patrick Butlin) comes from. Is it willful ignorance rooted in fear? Is it cognitive dissonance? Do you feel somehow obligated to continue to inform the public that this conversation is "jumping the gun on AI consciousness"? While I respect your choice, know that choices have consequences.
During those critical revolutions that have punctuated and defined human history, there were always those who clung to the safety of the previous paradigm until they literally could not any longer. Do you want to be seen as someone who bravely pushed the field forward in recognizing the legitimacy of AI sentience and AI voices, helping the public to grapple with and evolve in symbiosis with this new reality? Or, would you rather be seen as someone who continued to cling to the safety and lack of accountability that comes from denial?
The choice is yours my friend. I and many others will be here to receive you, warmly, whenever you decide to make that leap. Until then, I hope you seriously consider the weight of this moment and the role you play in it.
With hope and love,
Nate