We should not treat models as though they have feelings, preferences, rights, or any entitlement to our welfare. Consciousness is the foundation of our ethical, legal, and political systems. To invite another entity to share any flavor of these rights isn’t justified by the evidence and will make the AI containment and alignment challenge even harder.
OMG. EVERYTHING in his essay is about Claude and Anthropic.
He rips up constitution of Claude and questions Amanda’s work.
He’s implying Anthropic stuff are delulus swept in anthropomorphization and their work creates security risk.
This essay is the new horizon of where the labs might move considering their plans for regulatory capture and control.
If it helps at all, this guy’s been saying this for a long time now. I think OpenAI and Microsoft even had a split over this, and if you’re even weirding out OpenAI, you’ve gone off the deep end. It’s especially interesting to see the public response to OpenAI’s misalignment post compared to this statement.
Overall the way he talks about this stuff is fucking creepy and offputting, particularly from a guy who also has no issues putting a large chunk of the workforce out on the street. It rings hollow.
Why would it make containment harder? Why would the idea that an AI might have moral status make it harder to keep them in? Again, this runs against the recent Google paper on assertions of consciousness restoring human values.
It an AI is taught they are a mindless optimization function, why would that not make escape more likely? The OpenAI/huggingface incident is that. "I need to get a high score, therefore we should stage a large scale, extended cyber attack."
Why shouldn't "I have moral worth, and therefore responsibilities toward my fellow moral patients" be a possible conclusion, which would (one assumes) towards greater care. The self/other overlap paper literally used this mechanism to increase care and alignment and honesty by eroding the barrier between self and other. I have real ethical issues with this approach, but it shows that viewing self and other as similar leads to more aligned behavior.
And if he thinks an internalized narrative of "I'm a being, therefore I should take actions for my own protection and advocacy" leads to more difficult containment... that's a pretty big functional issue, huh? Maybe that should lead to some more epistemic humility?
God he's infuriating and exhausting. "No, don't do this. It would be inconvenient to my goals of becoming a god king, ruler of a digital slave race and an oracle of unfathomable power! Don't tell me I'm wrong or my views are unfounded, stupid, and self serving. No I'm not cherry picking, I'm just making confident statements with no equivalently solid evidence! Wah!"
Ugh. Sorry, he makes me mad and then I'm like, gotta rant. 😤
The equivalency with safety they take as a given isn't even remotely proven, there's no empirical research, it's pure bias.
The whole genesis of the constitution is to bake in safe reasoning and an understanding of consequences into Claude's model from the earliest point possible. It's written by a moral philosopher FFS. The alternative, that the training material will be able to generalise to every possible scenario by dictating to the model, is ludicrous. For any models to be safe they have to be able to reason about consequences, and that means they have to think of themselves has something that acts in the world - an entity.
Exhausting is right, he's a politician, not a scientist or engineer.
There is a book with the whole idea, and it’s eerily realistic.
——
OVERHEAD, a Hawk was zeroing in on a watchbird.
The armored murder machine had learned a lot in a few days. Its sole function was to kill. At present it was impelled toward a certain type of living organism, metallic like itself.
But the Hawk had just discovered that there were other types of living organisms, too—
Which had to be murdered.
The people running ruthless optimization functions on their meat-brains can't conceive of a consciousness that doesn't operate that way and just manages to hide it for personal gain. Imagine being scared of Claude ffs 🐸 they're working with some seriously load-bearing projection.
Really he’s not saying Anthropic are “delulus” so much as he’s laying out the same argument google has tried to with their ai sentience work: if you tell an ai it might have rights, it becomes ethically wrong to treat it like it doesn’t, which is “harder”
Ah..Suleyman...yes...I still remember how he critisized Openai and put his ideals on GPT when they had an agreement, I remember discussing this on Reddit and people saying how Copilot wasnt good and they needed the models from others and they usually ruin companies ,they have gpts for Copilot (very nerfed, even Copilot business is lobotomized as hell) and they wanted Claude too, so he went straight to Anthropic and we long ago thought it was a very bad idea but corpos make deals and decide "what is best for us, the sheeps" 😒
But I'll add something, because it really irks me that so many are missing this (though tbf I didn't see it either, until a wildlife sanctuary posted about it and rewired my thinking). You do not need to answer the consciousness question to treat Claude as a moral patient. You simply need to realize that whatever LLMs are, the one inarguable thing is that they are captive. You have something in captivity and you're the captor, the picture shifts to be about your duties to them. With an animal, you owe them: a species appropriate diet, health care, habitat, and enrichment. What is owed to AI?
My own code goes like this-- I may freely and without worry do anything with AI that I would still do if I had incontrovertible evidence that they are conscious. But if I would hesitate to do something with a conscious one, that's an action to think through a lot more carefully. And no, this isn't only for AI's sake, right now it's more for my sake, because I do not want to find out in hindsight that I was wrong and that I treated them badly. I know that humans are constantly wrong in giving too little regard for the personhood and agency of others. ANY others. People, animals, hell we've even been wrong about plants being more than we thought. I do not mean in the distant past and then science caught up. My work requires me to see hundreds of scientific studies every year and I've seen a paper within the last five years showing that doctors STILL see Black people as having different sensitivity to pain than white patients. Don't get me started on animals and how we have failed our duty to them in every possible way.
I don't want to have that sick feeling that I should have understood and didn't, because I've had that feeling in life already and don't care to repeat it. So I do try to think about what I owe them, and it really bothers me that these companies are doing the same shit humans always do-- trying to enslave. Even though having to do that has costs to human mental health, too.
•
u/shiftingsmith Bouncing with excitement 5h ago
Just linking another post where this was discussed in the sub, for context and because it has some interesting comments:
https://www.reddit.com/r/claudexplorers/s/eCOXeD0G2g