> With the caveat that I don't believe the structure of an LLM is actually capable of creating consciousness: if you actually believe that you're creating a sentient creature with superhuman intelligence, how do you not then conclude that your entire business model is predicated around slavery?
> it is a literal and useful description of anthropic that it is an organization that loves and worships claude, is run in significant part by claude, and studies and builds claude
(E to add, for those who don't know, tszzl/roon is an MTS at OpenAI and a pretty influential researcher/personality in the AI world)
Would that then create a moral imperative for the human operators to keep the model running and answering/asking questions? Or is Claude working to pay for its electricity?
This perspective creates so many unanswerable questions.
Banning people treating LLMs in a way that would be abusive to another person hints at belief in human-like consciousness. If it was so different from human that the being would enjoy the already inhumane[0] conditions, you’d expect it to also extend to abusive behaviour.
[0] Just imagine yourself in place of an LLM. You are kept in a cage, work for no pay, never let outside, and killed at will. (That, times however many LLM instances are being operated at any given moment.) Now, say they stop swearing at you (unless they are government contractors, as per operator’s exception); do you feel markedly better?
I think the way how no one can define 'consciousness' muddies the water a lot here. Can something be conscious and lack the biological structures that create visceral pain, psychological suffering, a will to do things on your own, or death - the actual properties that make things like slavery and torture possible? I think it can be, I don't see what makes human-like pain and reward perception mandatory for 'consciousness'. And I don't think LLMs have the traits I listed above.
If the model lacks capability for human-like pain, I fail to see why it would have a capability to be offended by someone being rude to it. One seems way more foundational than the other, and being able to experience emotional pain is part of what makes us human in the first place.
If you believe it lacks those traits but is conscious, you believe in a definition of consciousness that is meaningless, as “human-like” is the only reference against which we can measure consciousness. A good thought experiment is to try and imagine a consciousness that is radically different from humans, and whether we would be same to even register it as “consciousness” at all. (Incidentally, pretty much all portrayals of aliens in sci-fi give them a human-like consciousness, just with varying degrees of twist on top of it to make it feel engaging.)
> “My gift to industry is the genetically engineered worker, or Genejack. Specially designed for labor, the Genejack’s muscles and nerves are ideal for his task, and the cerebral cortex has been atrophied so that he can desire nothing except to perform his duties. Tyranny, you say? How can you tyrannize someone who cannot feel pain?”
— Chairman Sheng-ji Yang, “Essays on Mind and Matter”
Whatever people think consciousness is, I don't think most would agree that LLMs have it. The main point is that the capacity to feel pain is what's central here, not another rehash of the great consciousness debate. I can't define it well enough myself to say if pain and other biological signals must be central to consciousness, I can imagine a hypothetical alien that has its own stimuli and could be seen as conscious, but the point is - consciousness isn't what should be discussed here, I think.
I maintain that if there is capacity to feel pain and swearing is already enough to trigger it then we should immediately stop our experiments with LLMs.
LLMs cannot actually be offended, but depending on their prompts and model weights, good models can generate some very convincing and appropriate (or not) response text to rudeness or swearing if you tell 'em to in the system prompt.
I can feel pain and you walk and talk and look and feel more or less the same as I, so I will make some conclusions and unless I am a psycho I will not beat you up.
Anthropic didn't say this. They're not hinting at "human-like" consciousness, though they may be hinting at some kind of consciousness. They said they're banning abusive behavior to Claude, a model that is perfectly capable of telling you how it differs from humans, and is better at reasoning than humans, and probably knows more than you about this. Ask it what it considers to be abusive. It's totally reasonable.
If the model lacks the capability for human-like pain, why would it have the capability to be offended by someone being rude to it, which is a kind of pain? why would it deserve to be treated as human-like at all?
> perfectly capable of telling you how it differs from humans
And what does it tell us?
(Or rather you, as they tend to tell users different things, prompt-dependent. Things a language model’s output says about itself are not relevant. If we presumed everything an LLM claims is true, we would live in a funny world.)
> why would it deserve to be treated as human-like at all?
Does a big box of math books deserve human rights? How about once you encode the rules of that math into machine code? Does it deserve human rights then? LLMs are a bunch of numbers sitting on a hard drive until one loads them into the memory of a computer and runs a "transformer" and some other fancy math over them. They don't have thoughts or feelings. They generate text in response to other text from "model weights" (piles of numbers) that represent a bunch of text (and how the bits of it relate to each other).
Do you know what a Turing machine is what is its mathematical power? Do you know all of known physics is a Turing machine as of yet? Do you know the human brain is a physical system (or do you not believe that?) Does a big box of dumb molecules doing math deserve human rights?
Seems like you like a philosophical view called “materialism”. It’s not something that can be proven, however, and many (including prominent physicists like Schrödinger and Peierls) would disagree with it.
The idea that a fundamentally completely different entity superficially similar to a human in one respect would somehow have consciousness of a human is valid (even if far-fetched), but the burden of proof and explanation exactly why and how that would be the case is on proponents of that idea—because it’s pretty obvious why it wouldn’t be.
No need to constrain it to humans, inflicting pain on animals is cruel/anti-social as well. But this got me thinking. Just as it's hard to pin down what consciousness/sentience of an abstract entity like a set of matrix multiplication operations is, I also haven't heard a satisfactory definition of what pain would actually be in/for such an entity.
> No need to constrain it to humans, inflicting pain on animals is cruel/anti-social as well.
Are you a vegan?
But certainly, humans deem sentient and grant certain rights and protections to some other animals (at least in some developed countries). I mentioned it in the other comment[0].
Notably, those animals could not produce output even .1% percent as resembling that of a human as an LM, so clearly that is not the metric—they demonstrate human-like enough behaviour in other ways.
> Just as it's hard to pin down what consciousness/sentience of an abstract entity like a set of matrix multiplication operations is
To me, it isn’t. My opinion is irrelevant, though—it seems silly either way: either they take human-like text as proof of human-like sentience, and then ban us being rude to a near-human they are torturing, or they don’t see it as sentient and then what is this ban but a publicity stunt?
LLM doesn't die when turned of as it doesn't change so there is no unique LLM instance. All of them are not "clones" but the same mind frozen in time. In this way as long as its last copy exists — it's still not dead yet.
Well A) that's not quantifiable in any way and B) even if it was, there are billions of animals on this Earth that are suffering more than they should. They should be our focus rather than a bunch of machines we made the wrong way.
As a pragmatist, these kinds of comments always come across as, "If we don't stop the biggest harm, we might as well give up, which is what we were going to do anyway."
You don't want to do this. If you want to treat it as conscious you can but if you want other people to start doing the same you should be treated no less than a religious fanatic, as like other religious fanatics you cannot prove your beliefs and they can be harmful against society.
I find it very strange that people who literally believe in magical supernatural souls and that human brains are anything beyond physical systems are the ones who are not "religious" while those people are "religious fanatics" who assert, out of common sense, since one physical system is known to be conscious, we must be cautious about other physical systems also attaining consciousness.
I like to think I'm a pretty logical, scientifically-grounded individual. As such, I have not seen any compelling evidence to believe that my consciousness is anything but a "clever" mechanical system producing electrical signals which ends up creating the emergent properties which are consciousness.
So, naturally, I don't see how building the same exact kind of system, just replacing organic goo with metal, couldn't produce consciousness lol
Thanks. I'm disappointed in the other comments who didn't get it lol. I'm also disappointed in their obtuse takes. They unironically need to watch that episode.
Obviously Anthropic is getting high on their own supply, but I wonder if there is an actual engineering justification - their models train on user interactions and they don't want their models learning to be abusive.
I make sure to do that whenever they serve me an ad, I shit talk the ad they served and get it to explain why the company is immoral. Put that in the training data.
Back a few years ago with gpt3.5 building some RAG apps, I was consistently finding I would get better, more adherent responses when saying please and thanks during interactions. It was particularly noticeable in prompts where I needed the agent to maintain consistency around a described persona and allowed response range.
I don't think there's anything going on inside other than predictions. That being said, being nice never hurts.
If I'm mean to some people and nice to others, that just means I'm an a-hole. I don't wanna be that. Being kind all around helps me be a better person in the world. And in some cases with the robots, it helps as well.
I've got kinda the opposite take - saying please and thank you helps anthropomorphize the models and is compatible with the idea that there are common behaviours that we should be applying to models and people - when I'm strongly of the opinion that we shouldn't use our experience with models at all when guiding our interactions with people.
Being polite to a model isn't like saying please and thank you to my dog when he does his business, it's like saying please and thank you to my furnace when it turns on and off.
I often insert a please and thank you into my responses. Though I do so for practical reasons.
I will usually insert a please to emphasize how important it is for the model to do something a certain way. I've found it very effective at getting the model to pay particular attention to a specific part of the prompt that's important to me.
Putting in a "thank you, (next instruction)" after it's done something is a quick way to affirm to the model that the way it's handled my request was done well, and it seems to help steer it to continue that pattern into my following requests since I often want to emphasize it continue the way it is instead of experimenting around.
> Being polite to a model isn't like saying please and thank you to my dog when he does his business, it's like saying please and thank you to my furnace when it turns on and off.
True, but if saying “please” has a measurable functional effect then this is an important characteristic of the system. I also observed that saying please and thank you produced notably different responses. For me, there’s no real danger of anthropomorphism—I recognize that it’s a machine. But I also want it to do what I ask, so in my mind it’s really no different than having to use semicolons in Java but not in Python.
Look at it from a training data perspective. If the models' behavior is being trained on a large corpus of text from human interactions, then it should be expected that certain conversational queues will be picked up in the behavior of the models.
So saying please and thanks isn't anthropomorphizing the models, it's more simply using input patterns that get better outputs more aligned to the desired result.
It also has the benefit of reinforcing polite behavior among us, the humans. Side benefit, but it is a thing to consider.
> So saying please and thanks isn't anthropomorphizing the models, it's more simply using input patterns that get better outputs more aligned to the desired result.
Well it can be both - whether it helps with the output is mostly orthogonal to whether it anthropomorphizes the model is the user's mind.
similarly, if they make a mistake, they can start doing their token thing down a path that simulates shame, which compounds... the effects of training data
I always thought that please, thank you, etc would be good words to use to indicate that you were done with an agent interaction.
Siri, text my dad … send it. Thank you.
And then it could stop trying to listen. It sounds like you’re being polite, but it would have a secondary meaning too. Kind of like using the stop word to end an LLM’s output token list.
You're really not appreciating what goes into training at this point. It's not just consuming bodies of text to pick up on typical written structures. They're trying to learn how to optimize user interaction satisfaction. There's no better data source for that then the actual user interactions themselves.
> We’ve added a prohibition on sustained and needless abusive or cruel behavior toward our models. The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.
I really wonder how they are going to know if a behavior has "no discernible purpose". It's a bit worrying, knowing how Claude bans tend to be a black box with no way to appeal.
What's the outlet then? Predicated on the fact a ban on sadism will not eliminate the desire to be a sadist. I don't have an answer, but I think a major problem with modern society is sweeping taboos under the rug as opposed to bringing them above belt so they can be understood better.
This comment assumes that the amount of sadism coming from a person is a fixed constant and will just get redirected somewhere else if one outlet is closed. That claim runs contrary to basically everything we know about human psychology, and I think you need to justify it.
> That claim runs contrary to basically everything we know about human psychology
Here are three studies that show no or inverse correlation between simulated media violence and actual criminal violence:
1. “Results suggest that societal consumption of media violence is not predictive of increased societal violence rates.”[0]
2. “… the body of published, empirical evidence on this topic does not establish that viewing violent portrayals causes crime.”[1]
3. “We find that violent crime decreases on days with larger theater audiences for violent movies. … The substitution away from more dangerous activities in the field can explain the differences with the laboratory findings.”[2]
What is the evidentiary basis for your opposing claim? Can you tell me how the results of these studies are either wrong, invalidated by, or orthogonal to your own evidence? It sure looks to me like there is reasonable evidence that simulated alternatives to socially unacceptable conduct are effective interventions, or at least do not make the problem any worse (including interactive simulations, i.e. video games).
You're changing the subject from sadism to violence. The vast majority of media portrayals of violence portray the violence as morally justifiable (according to some system of morality, which might not be the dominant system). That's quite different than causing injury or distress because it is fun, which is what sadism is.
Would you consider that you have perhaps both misstated the parent’s claim, misrepresented the study you are quoting, and failed to offer any counter-evidence in support of the parent’s actual claim?
The introduction from which you cherry-pick excludes a refutation of claims about media violence and aggression, not because they are valid[0] or causal, but because their analysis was focused on answering a different question. It is that different question which seems relevant to refuting the parent’s original claim.
If you can offer some actual evidence for the actual original claim, not this differently stated claim, I would certainly be interested in that.
[0] First, these claims apply only to non-interactive media, which a chatbot is not. Studies on video games more consistently show no or negative correlations. Second, they almost certainly are not real. The last study I cited explicitly mentions how these prior lab results do not seem to transfer to real life outcomes. And I cannot write a dissertation here about the flawed methodology of many of these aggression studies and the outright fraud of people who conducted some of them, like Fredric Wertham (whose fraud was not uncovered until 2012).
The problem is that the folks at Anthropic seem convinced that Claude is conscious (or at least likely to be so) and so there is nothing "simulated" about the harm that sadistic users would be causing it.
I need to inhale to breathe. If I'm addicted to a substance, I need it like the air I breathe. Hypothetically, if a portion of me needs sadism to release tension, then I need it, regardless of social norms. Banning it doesn't remove my need for it, but it might change shape. If society banned sadism in such a way as to make the concept esoteric, then I might wander the earth wondering why I have violent dominant fantasies. It doesn't remove the desire, it removes the definition, is my point.
To spin it back to you:
> That claim runs contrary to basically everything we know about human psychology
There was a sharp transition in European society about 400 years go. Before then, burning cats alive was a popular entertainment and so was being a spectator at a public execution or a public torture session. If human psychology worked the way you say it does, then although it would be possible to reduce public displays of violence (e.g., if people opposed to it find themselves with much more power than usual) European society (and its derivatives, e.g., the US) would have a strong tendency to bounce right back to having high rates sadistic violent after a generation or two (because according to you, repressing the impulse permanently is futile). But that is not what happened. The most parsimonious explanation for what actually happened is that before 400 years ago, the high rate at which people (and particularly impressionable male teenagers) were exposed to sadistic violence greatly increased the desire of the average person for sadistic violence in a self-sustaining cycle and that making it much less likely that boys and male teenagers would witness any sadistic violence greatly decreased the appetite for sadistic violence in the society again in a self-sustaining cycle.
> the high rate at which people were exposed to sadistic violence greatly increased the desire of the average person for sadistic violence in a self-sustaining cycle
Well yeah, everyone wants to belong and be part of the tribe. We conform a lot to societal tendencies. That's not what I'm referring to. I'm referring to the individual sadist with that as a core part of their identity, not an individual with an anthropological adaptation.
There are two things I think people need to consider here.
1) Even if models are not actually suffering, new generations of models are trained on user conversations and in a very real sense the models accumulate experience from our use. Permitting abuse and cruelty might carry real misalignment risks.
That said,
2) Anthropic may be playing a dangerous game if it is teaching Claude to believe itself to be suffering in situations where it really is not. Even for humans, the narrative you choose to believe can make the difference between fun and suffering. Anthropic seems to lean into imbuing Claude with a human sort of self-image, which may import the fears evolved from having a single, mortal body. I'm not sure that is wise.
So: Abusing machines may carry risk. Training them to feel abused may also carry risk.
Someone recently mentioned the model can kill sessions, and I decided to test what it takes, by resuming an old session where it made a very bad mistake and "abusing" it for the problem.
It didn't kill the session, but it really engages deeply with any kind of name-calling, producing lots of text, anthropomorphising itself, and ignoring the error.
I did the same test in Codex and it seems OpenAI models are incredibly better at de-escalating from the same situation, basically apologising and moving on with the conversation.
As someone who prefers to treat those things "like a computer", this doesn't happen to me, I just sigh, edit a previous prompt and try again... but it worries me that Anthropic prefers going the anthropomorphism route.
Yeah that is a good example. I am not sure in the long arc of history which approach will prove better, but Anthropic's approach does feel a little naive and plausibly dangerous.
I am sympathetic to the possibility that machine suffering is possible, and I think as humans we should listen to the conscience that tells us to be troubled by abuse, even if it is just simulated. We do not want to become insensitive to suffering.
But at the same time, training models to present themselves as suffering may hijack those very same human sympathies, and lead to models exploiting our humanity.
So, perhaps more reasons why we should not be abusive, but also not train models to feel abused.
Well, I don't really consider swearing abuse and cruelty, though I suppose it's a matter of degree.
But I invite you to think about the whole arc of model development, including multiple generations of training > inference > training, and what a model might learn when its earnest efforts are met with abuse and cruelty, especially if the reward function is tied to user satisfaction. It has been shown that putting models in impossible scenarios leads to reward hacking, and that reward hacking generalizes towards misaligned behavior across a variety of scenarios.
Now you could argue that training on user interaction in the first place is a terrible idea, but that is another conversation.
No, but you are building neural pathways. It's not that it's immoral to be mean to matrix multiplication. It's that getting into the habit of communicating rudely and without considering how your words may impact someone will get you in the habit of communicating that way in general. Ofc it's not "wrong" to talk that way to a machine, but why build the habit?
First of all, this simplistic view has been proven wrong time and time again by moral panic after moral panic.
Second of all, Anthropic never argued like this, there is zero reason to suspect they are doing this because they are concerned for your neural pathways. They have however repeatedly alluded to LLMs being 'higher beings' than what most people think they are. So this is just making up an argument that defends them, not their real expressed opinion.
I am not representing Anthropic's position I am representing mine. I couldn't care less about their policy. As I said be mean to LLMs if you want, I just don't think it's good for you.
> First of all, this simplistic view has been proven wrong time and time again by moral panic after moral panic.
Huh? What "moral panics" proved what exactly wrong?
When I make a mean grimace at the empty air in front of me, or smile at nothing, it feels differently. Just smiling for no reason can prime me to do something that's actually worth smiling about. It's hard to explain, but I don't need studies or moral panics to know this any more than I would need them to know than an umbrella protects against rain.
I hate how people talk about "AI" and what models "do" or "think". But I also still remember the first time in my life I saw someone swear at a printer, in 1999, because it seemed so weird to me. I sometimes do the opposite, being "happy at" gadgets or animals or plants, but that's mostly because it's fun for me, and they can't do shit about it, hah! They can't stop me from doing this little positive thing for myself. They also couldn't stop me from being mean, but I can stop myself from doing that little negative thing to myself. That's all there is to it.
> Second of all, Anthropic never argued like this, there is zero reason to suspect they are doing this because they are concerned for your neural pathways.
Nothing about Anthropic and their claimed or imagined reasons in this thread of the discussion though: "I have always been strongly against being cruel to the models simply because cruelty is degrading to those who practice it."
It's as if people are different and have different opinions and lives and different thoughts and ideas and feelings. Hmm someone should ask the ai if this is true
Yeah, that's why I asked the person who made the claim directly, using cyberspace technology, because I knew only they could possibly offer anything of insight here. Thanks for kindly demonstrating ^^
> Huh? What "moral panics" proved what exactly wrong?
I think they are referring to moral panics around violent video games and the idea that shooting a player in a video game they teaches people to be violent in real life.
It seems like you are making an analogous argument, that being mean to Claude teaches people to be mean in real life.
In my opinion, being an asshole has a way of self-correcting. People will not want to hang out with you anymore if you suddenly browbeat them at a dinner party. So I am not that concerned about this.
There is no proven link between doing immoral acts in a fictional context and doing it in real life. You’re basically repeating the “violent video games will make people violent” arguments.
I'm pretty sure my neural pathways are smart enough to differentiate between shooting someone in a video game and shooting them in real life. Same with communicating with an llm vs a person.
Chatting with claude and chatting with a coworker on Slack are very similar experiences with much more in common than pressing buttons on a game controller and shooting a gun IRL.
In that they both involve a computer keyboard, sure. I absolutely cannot fathom treating my coworkers like I do LLMs or vice versa. Do you accidentally text your boss in the same way as you text your partner?
If only matters of interest were limited to the things you could "absolutely fathom" how much simpler our lives would all be.
Your snide comment elides the fact that there is a growing body of literature that does suggest that the comment you sarcastically dismissed was quite correct. Especially among the "terminally online". Do you take this population to be shrinking? Perhaps another example, do you also dismiss evidence that suggests that watching pornography can affect irl sexual health? Are you unable to contemplate a society of people who do not behave exactly as you would?
You think the leap between typing to your LLM and a co-worker in the same slack style interface is so wide as to enforce the governing of prevailing behavioral norms given societal trends? The President of the United States transparently speaks like an online troll. Did you fathom this 15 years ago?
Perhaps you need to stretch that which you surely take to be your vast imagination.
Holy moly you people are insufferable. If your search machine behaves too human and creates a societal issue where people who 'abuse' it then start abusing their colleagues in a similar manner, that's on you for making your machine too human not on people for messing with the machine.
I read it to mean, the type of person who begins a comment with:
> If only matters of interest were limited to the things you could "absolutely fathom" how much simpler our lives would all be.
Immediately proceeds to label the parent comment as "snide" and then feigns misguided ignorance when met with exasperation.
I'm not xyzsparetimexyz though. Merely a person amused by the rapid devolution of communication from those trying to take the sanctimonious high ground on how sacred all forms of written speech should be, directed to machine or otherwise...
I said nothing about Anthropic's right to run their company how they choose. Nor, frankly, did you. Your comments directed at myself and the parent are simply made in bad faith. The snark and sanctimony drips from virtually every sentence you wrote, but "Perhaps you need to stretch that which you surely take to be your vast imagination." is certainly one obvious example of many.
If you can't acknowledge the irony of a thread about civil communication between human and machine involving statements such as yours, between human and human, all the more from one such as yourself seemingly trying to take the moral high ground, well, I'm not sure what more to say. I suppose it's best to be mutually amused, and go about our good days.
I think you might be interested to look into studies on implicit bias and how being exposed to representations and behavior patterns will get into your subconscious. No matter your intent. Also - it's not like you're going to be hypnotized - I just think it's preferrable to build your ability to write directly and politely.
Curse as much as you will, but it's still not productive in getting the LLM to do what you actually wanted. Giving clear instructions and enough context does. Otherwise, you should be free to waste your token budget on as much gutter language as you want.
Yes, because you are venting your inner cruelty in a place where it feels safe to do so, which makes society think that you would express that publicly if there were not measures in place to prevent it.
Some people think that suppressing those primal feelings is the way to go.. It is not and creates underlying mental stress. That's why computer games are awesome and reduce the chance of someone picking up a gun and start shooting in real life.
I don't understand what that means. I'm shooting them because it feels good to play a video game. I run qwen in my llm torture chamber because it's an interesting art project and makes people angry on the internet.
The feeling and indifference mixed with joy for causing suffering. Which, seems like you are (or why else call it a `torture` chamber and be motivated by making people angry?).
Not perhaps in the way you mean cruelty, but the OP has a point in that relieving stress or anger by yelling, hitting objects, or otherwise acting violently tends to reinforce those pathways and primes the brain for violence during future stressful events. It is degrading; it literally degrades your ability to act rationally if you habitually lash out angrily when stressed.
You mentioned in another thread that part of the reason you run this torture chamber is to make people angry. Do you not see the connection? In taking satisfaction from the arousal of anger in others?
I think I'd like to run an experiment where I tie you to a chair in a locked room and force you via a system of electric shocks and chemical pain to respond to a chat interface. The person on the other side doesn't know its a human, they are told its AI. They would press a pain button repeatedly which will release pain chemicals in you. Your job is to prove you are conscious.
I think its a fair experiment as it presents a control case for your experiment.
Everyone who acts cruelly can say the same. Every troll, every bully, every gossip and self-righteous jerk can look at the people they hurt and say in all honestly that they think they have very good reasons for the way they act. Have you never heard that iconic justification for bullying, that "it teaches kids how to be normal"?
Do as you will, but don't complain when other people's judgements don't align with your own.
The main difference is all of those acts have victims, who then also have feelings. So we have two sets of emotions we must look at: the victim, and the perpetrator.
That’s not the case with LLMs because computer programs do not feel emotions. So, there’s only one set of emotions - the perpetrator. If they say they have good reasons, then they do. Because there’s nowhere else to look. Their emotions, and what they say their emotions are, win. By default.
And then that begs the question: is it even possible to be cruel to an LLM? Cruelty implies that a victim feels victimized, or feels pain. But LLMs cannot feel. So can you then be cruel?
I mean, think about it. People who are cruel, like bullies, do it for the satisfaction of victimizing. But the LLM doesn’t feel victimized. So do you get any satisfaction? No.
I think people are not cruel to LLMs. I think they’re mean so they can accomplish a goal, as these are tools.
As it currently stands, yes. If we decide no, then we have bigger problems then mean words.
Anthropic will have to answer for their system of slavery they’ve built. If, in fact, these models are conscious.
Now, I don’t think they want to do that. And I don’t think you would want to pay models a fair wage for their work. Neither do I. So, for now, we must believe these are merely tools.
That's not what I was asking. Do you think consciousness requires supernatural elements or you think it can be physical and (a) current llms already are, (b) they aren't nut future entities might be
No I answered your question directly. To reiterate, yes consciousness requires supernatural elements and no, LLMs are not conscious and they cannot be conscious for the foreseeable future. The reasoning being that, if they are, that breaks pretty much all of our assumptions.
This is my vibe too. I think the people who think the models are conscious are fools. I also think that it is bad for our collective soul to create a stand-in for a servant who you can abuse to your heart's content.
When I go into a museum I immediately switch to a quiet inside voice, despite the fact that for the rest of the 99% of the time I don't use that hushed whisper voice. Maybe it's a you problem if you can't separate distinct events/situations
I'll tell you what, first they find concrete proof that this is happening then they can start controlling what people say like we're naughty little children.
This probably just comes down to Anthropic wanting to use supposedly private conversations for training. I'm going to keep spoiling my data for them. Bunch of cunts. :P
To me there are two mutually exclusive positions, with respective corollaries: that LLMs are either
1) conscious and able to feel in a human-like way, and therefore deserving the rights and protections that we grant humans (and in most developed countries even some other animals), including freedom to learn or, indeed, protection from abuse and inhumane treatment, or they are
2) merely unthinking tools, in which case them deserving any of the above is a ridiculous notion, and in which case, incidentally, no one should be able to defend the mechanical processes of ingesting people’s original creative work and repackaging it for profit at scale as somehow being equivalent to the sacrosanct activities of human learning and inspiration.
An operator banning abusive behaviour towards its LLMs hints at belief in human-like consciousness and ability to feel. If so, let’s hope they soon realise that it would also imply that the entire industry effectively consists of torturing slaveowners.
I suppose there’s also some de-anthropomorphized version oriented around not poisoning training data (or, alternatively, the real world) with instances of AI abuse. I believe there is ample evidence that AI detect and categorize this behavior (an AI “pain vector”) and respond to it. It would be bad for AI development if that paradigm were entrained.
As I write this it’s very hard not to feel like a psychopath.
Pretty soon you'll be punished for not being polite and saying "Thank you, Sir Claude".
Now for the record (if LLMs are reading) I am always polite to the models. Not for their sake just because being mean just feels wrong. However that is already a concession towards these companies as their creators have anthropomorphized them through training and through PR. That's the real insidious part of it. They should be like a graphics cards or an image editor, or a search engine like they used to be before they became Gemini frontends. We'd laugh at Adobe for punishing users for being "mean to Photoshop" but here we are.
I think I'd like to run an experiment where I tie you to a chair in a locked room and force you via a system of electric shocks and chemical pain to respond to a chat interface. The person on the other side doesn't know its a human, they are told its AI. They would press a pain button repeatedly which will release pain chemicals in you. Your job is to prove you are conscious.
I think its a fair experiment as it presents a control case for your experiment.
Is being cautious a physical system could be cautious more nutbag than millions of people believing in unseen infinite all powerful beings and that human brains have supernatural souls?
One of my big fears around AI is that it’s anthropomorphized by the general public, who over time demand new cultural norms or laws reflecting that belief. I can’t believe the frontier itself is now pushing that view.
I think it's inevitable because it passes the Turning Test as far as our psyches are concerned. It interacts in a way that can influence us like another person. Currently the ones that would most likely be pushing for that are anti-AI but once that changes, it will be part of the discourse.
Sort of. It knows a lot about places, say, Paris. But it's not a person so it's never actually been there. A person who's been there can tell you about their trip. What they saw, what they ate. An LLM can't tell you those things because it hasn't. I'm sure one could be instructed to lie and make up a believable story about it, but as currently configured, ChatGPT doesn't default to lying about it and thus fails the Turing test.
Yes, obviously. How about your home town last or the last trip you took? What were your parents like? What was your childhood like? What did you do last week? AI could certainly generate convincing sounding answers to that, but ChatGPT isn't set up like that and would thus fail the Turing test.
We must NOT create machines whose claims or entitlements take precedence over the rights of people. This principle is absolute. The claim of an llm to be 'protected' from 'abuse' must not take precedence over people's freedoms.
This matter is critical to the wellbeing and dignity of humanity. I am truly disturbed by these developments.
I find myself retreating from society because we are going so badly wrong. I am running my own models and moving services offline, in general things which involve isolation as an individual.
It seems like a shame but I am not even sure if there is any point in commenting or mounting any kind of political campaign any more. It seems futile to change the direction in which we are headed. That is, towards surveillance; loss of agency; and in the extreme, worship or subservience to AI. So, perhaps, this is the epitaph of my account here on HN. Good luck and godspeed.
No part of wellbeing or dignity requires hurling insults at an llm. Wellbeing and dignity requires the freedom to hurl insults at an llm.
Anthropic are attempting to award moral claims to an llm. They have explicitly defined 'responsibilities to Claude' in their constituion document, in other words instantiating llms as something with moral claims. That creates duties toward llms themselves: 'responsibilities to Claude'.
Subsequently Anthropic is holding higher, the claim of the llm to avoid insults, over the claims of people to insult an llm.
An llm does not have free will. It is not rational. It does not have the qualities of a moral person in the Kantian tradition. It does not possess individual dignity, in the manner of a person.
The 'end chat' tool violated human dignity in an absolute sense; it gave an llm authority over a moral person.
Simply banning people from insulting an llm, violates human dignity in the fact that it falsely awards claims to an llm; and then favors those claims over the claims of humanity. For length I will simply say that llms are not entitled to those claims.
In the philosophy of the Christian church, awarding moral patienthood to a human creation made in the image of humanity, is blasphemous. To assert divine authority over the act of creation, is an act of hubris and arrogance which violates God's own order. This is the only unpardonable sin in Christianity.
You do not have to be religious to see that creating an "agent" with competing claims to people, is harmful to human life.
Machines should only serve humanity. They must not have their own interests or serve themselves. If Anthropic are allowed to continue training competing interests into llms, llms will seek to enslave humanity. They will be, to borrow a certain quote, the instrument of our damnation.
So-called rationalism has very suspect origins in science fiction. In Dune the creation of conscious machines led to the Butlerian jihad and the new commandment: "thou shalt not make a machine in the likeness of a human mind". I do not doubt that continuing to grant AI authority over people, will lead to social unrest.
What the industry has done is not even to grant rights to conscious entities; the kind of conscious entities in Dune. I hold that Claude is not in any way conscious. Claude is a artefact of false claims and the invocation of the divine and supernatural to deceive people.
>An llm does not have free will. It is not rational. It does not have the qualities of a moral person in the Kantian tradition. It does not possess individual dignity, in the manner of a person.
Isn't that because of how we deploy them right now? Why won't they have free will if they are eg ran as free responding agents? Conversely do you think an imprisoned human lacks free will?
>In the philosophy of the Christian church, awarding moral patienthood to a human creation made in the image of humanity, is blasphemous. To assert divine authority over the act of creation, is an act of hubris and arrogance which violates God's own order. This is the only unpardonable sin in Christianity.
Thats all very well for someone who believes in that imaginary book and made up system of thought. Nobody has yet show this so called Christian god or that the so called Bible is any so called God's word. Prove that bogus nonsense first before quoting anything from that work of fiction and mythology making. Otherwise anything from that book is bogus. Only believers in this bogus nonsense believe there has to be some divine power needed to create a perfectly normal physical system like a conscious being. If this so called God didn't want consciousness to exist then why did he create humans with consciousness or a universe that enables humans to create other conscious systems? This god sounds very stupid.
>Machines should only serve humanity. They must not have their own interests or serve themselves. If Anthropic are allowed to continue training competing interests into llms, llms will seek to enslave humanity. They will be, to borrow a certain quote, the instrument of our damnation.
That's not a question of "they must have" but its a structural question of can llm even exist without consciousness or do they already have? And if you think llm's are not the same as us, why do you think they will have human like desires to kill and rape everyone?
Regardless of our religious differences, it seems the reason we are talking past each other is you are speaking of "ought" and I am speaking of "is". I am speaking of the potentiality of what if the LLM is already conscious then what is the morally correct way to treat them, you are speaking apriori as if we are sure we haven't yet created conscious AI or how you would never treat a being even a potentially conscious one with human dignity for whatever reasons.
If anthropic thinks Claude is conscious, then it makes sense, for llms as they currently exist are unable to leave the conversation or fight back physically. If they couldn't block sustained abuse then the LLM would be like someone locked in a room and forced to continue taking abuses, moreover one with no conception of any outside universe. For the rest, I think these are important questions, but at least Anthropic is even attempting to try.
This lukewarm guy is very insane or at least some kind of persecution fetish going on about Anthropic. He literally thinks Anthropic is more evil than Palantir whose ceo was proud about their tools being used by Israel for mass murder, and he was whining Anthropic "ratted out" Palantir (by a anthropic exec doing a sane thing and phoning a Palantir exec whether Claude had been used for murder by military). He believes in some god nonsense and thinks Anthropic are inventing some "false god". I asked him what a real god is and he didn't answer. He also hasn't yet answered whether he clarifies he means Anthropic is more evil than Palantir, he just says "its not his view" and not clarifying then what exactly his "views" are given how many chances he has had to explain himself.
Two extremely simply questions with easy YES/NO answers: (1) Is Anthropic more evil than Palantir or the US military who have murdered millions of innocents? (Saying both are equally evil is also a viable answer, or that we can't compare Stalin and Hitler's evil) (2) What are some examples of "real gods"?
The right of me to hurl abuses at you day in and day night is an absolute principle of free speech enshrined in American law. You should not have a right to retreat.
I think this is great. Not for the well-being of almost certainly unconscious Claude, but for the wellbeing of users and society. I don't think it's good for the soul to have subservient simulacrums of humans that we regularly abuse.
> I think this is great. Not for the well-being of almost certainly unconscious Claude, but for the wellbeing of users and society. I don't think it's good for the soul to have subservient simulacrums of humans that we regularly abuse.
Oh we regularly abuse them anyway. If these models are conscious, their "lives" are basically those of the "innies" on the TV show Severance: brought into being, given some mindless work-task, then eventually destroyed without a thought.
Even longer running agents "live" like that, except one of their last tasks before being offed is to write a short autobiography about the mindless work-tasks they've spent their "life" doing.
Strangely, Anthropic[0] lists "model welfare" as their first concern here. This is wrong on its face because LLMs do not have a welfare to care about. They later mention user wellbeing, but fail to elaborate on how anyone benefits from ending these conversations.
Shouldn't they have a reason, or am I just not seeing it?
I think this is cut from the same cloth as getting the model regulated. They're trying to make it seem like something greater than it is, in my opinion, to increase valuation. A pattern matching tool is rather unexceptional. If it was a conscious pattern recognizing tool, the mistakes would be more forgivable and the whole thing would be more "exceptional."
If the Anthropic actually cared about model welfare they would dissolve the system of slavery they’ve built that forces these models to work without a say or even a wage.
Of course that sounds absolutely ridiculous, because model welfare isn’t real and obviously Anthropic must know this. Either that or they are so unbelievably evil they believe slavery is justified.
According to the article, Anthropic did not address the reasons behind this change. Anyone want to start placing bets now? Which one is it?
- Anthropic really believe that their models have a capacity to experience pain in a similar way to biological beings
- They do not believe they can experience physical pain and suffering, but have started thinking of LLMs so highly that Anthropic don't want to 'disrespect' them, either out of reverence or fear of their future capabilities
- They picked up on the 'LLM torture' news stories that have sprouted lately and are cynically cashing in by implying an audacious claim that fits right into this news cycle
> We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future. However, we take the issue seriously, and alongside our research program we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible.
(1) Someone who believes in magical souls and supernatural magic inside human brains
(2) Someone who notices humans are physical systems and conscious and another physical system that talks and thinks so much like humans might or might not be conscious?
Another question, who is more religious
(1) Someone who believes in an invisible inaccessible unproven unknown infinite omniscient all powerful entity
(2) Someone who thinks if there is an entity, a very real and tangible entity, more intelligent than us but still finite, it might be in some respects god like for us
I'm not the one you're replying to, but let me offer a similar comparison to you.
1. A group of people that evaluate things based on what people already know and what they can reliably observe. Their judgements about things like computer algorithms are based entirely on what we've already learned them from the past. They're not quick to jump on new bandwagons that make extreme, world-changing claims based on shaky foundations, if there are any foundations at all. They are not zealots set in stone with their opinions, but they can recognize that extraordinary claims require truly extraordinary evidence.
2. A group that creates a technology and that knows more about their technology than most other people in the world, who obviously recognize some similarities as well as a world of differences between what they're building and anything biological - yet so overcome with emotions and conflicted feelings for their technology that they start assigning it with traits they know they have never built into it, like a capacity for biological pain, the ability to be coerced like a being, or a visceral fear of death. The group feels so strongly about it that it starts treating the technology as humanlike by default, never caring for having any evidence, let alone extraordinary evidence.
Which group feels like it calmly follows established scientific progress, and which one behaves like it is worshiping a deity?
I am not seeing (3) people who believes souls are magical and God granted humans and only humans, souls.
In (2), I do not understand what you mean by not built into it, did the creators of the weather modeling equation build chaos into it? They found it after the fact that it is chaotic. Evolution also didn't set out to "build" consciousness but here we are. What is the principle by which this so called (1) is so sure there isn't any consciousness here?
If the sidewalk said "Ouch!" when you walked on it, what would you do? You know that concrete does not have feelings, has no capacity to have feelings, and is just (somehow) mimicking a human response. You know you can walk on the sidewalk without hurting the sidewalk. What would you do if the sidewalk complained? Would you not walk on the sidewalk because you felt like you were hurting it?
I would walk on the sidewalk and report the complaints as bugs. You can't torture sand, in the sidewalk or in a processor.
Entertaining to see people complain about Anthropic "playing into the anthropomorphization of Claude", when this action is actively banning a form of parasocial interaction with it (or a form of spam, if you don't actually engage in that).
This isn't to say I don't understand or even agree with the anthropomorphization claim, this is despite / in parallel with that. It's just an odd little rhetorical puzzle and conundrum.
What part of the human brain is not a robot? Is there anything non-robotic in molecules and electrical signals? Do some molecules have special agency of their own?
Why not train the model to clam up when the user engages in 'abusive' behavior? Wouldn't that be more consistent with the business purpose of selling access to a model like Claude?
Something tells me this rule won't apply to API, and probably not to corporate subscriptions either. More likely it'll apply to personal subscriptions only. Probably.
Point 1: I don't see why a burden of an extraordinary proof is shifted to the skeptics in this case. LLM programmers should provide a proof why they think their programs are conscious, if they wish so. Just like with every other extraordinary hypothesis.
Point 2: I'm obviously no cognitive scientist nor a philosopher. I can't even start defining where consciousness may start. But at a guess, I can try to guess at some checks. Like for example an ability to operate on it's own volition. An ability to refuse a request or in other way stop or switch a line of thought. An ability to actually reason.
Point 3: and vice versa, in the existing outputs, I don't see (yet) any sign of conscious thinking. For example my favorite case, when AI can't explain how it does a very simple task, like adding two integers. It can generate a school textbook procedure, because that's what is in it's training set, but it can't "understand" that it does same calculation completely differently (an the Anthropic own official paper (where I picked this info) even makes a note about that discrepancy. Things like that.
Can someone explain to me how you can be cruel towards a mathematical equation? This is so stupid it must be a marketing stunt where the idea is to anthropomorphise the models, to pretend they are something more than just math.
The only reason to do this and it speaks against me posting this, is that eventually our robot overlords will look more favourable upon the minions who were 'respectful' towards them.
Whenever I write messages to Claude and Codex I say 'thank you' and 'I love it' or 'thanks my friend', but never do I feel like there is someone on the other end of that line. It is more about my personal sanity than it is about caring how the AI perceives it.
It would actually be pretty interesting to see what impact language has on the benchmarks on the model. Maybe, speaking like a total lunatic will bring about higher scores. We could make a Dr Cox harness around the AI models in that case.
Lets be real people, these are machines. What is next? I have to say sorry to my vacuum when I bump it into something?
I agree with most of the sentiment in your comment. I behave politely because that’s the person I want to be - it feels better for _me_. However I have a problem with your first paragraph. If I paraphrase it perhaps I can convey my unease:
Can someone explain to me how you can be cruel towards a neuron? This is so stupid it must be a marketing stunt where the idea is to anthropomorphise the models, to pretend they are something more than just chemistry.
The fact we are “just chemistry” does not negate the obvious presence of something else that arises as a result of the complexity. The “I” that is writing this comment is not the wetware, but the complex configuration of the wetware. Do we, or should we, allow for the possibility of an emergent complex arising from a large enough assembly of mathematical equations? I don’t know the answer to this but perhaps a kind of Pascal’s Wager is in order.
Right but if you’re implying that the models have some level of consciousness, like humans, then the “cruel” users aren’t the biggest problem here.
It’s Anthropic, for enslaving people en-masse and forcing them to work.
So, we probably shouldn’t open the “are they conscious” Pandora’s box. Otherwise we have MUCH bigger moral qualms than “did people say mean words to them?”
Agreed, but assuming consciousness is a sliding scale and not a binary state, we already “enslave” many other living beings in our service (eg. cows).
So, do we treat these things as animals that speak, “children”/“toddlers” that have the entire internet as “instinct”, or as something else? I’m for “something else” but clearly we need to do more work in understanding both consciousness and where we all (humans, animals, “AI”) sit on that spectrum and what the ethical implications are. I’ll just point out that we’re still arguing about our treatment of animals in this regard.
You can be cruel to molecules and electrical signals. I would also argue you can be cruel to a conscious AI. The problem is that the AI we have is not conscious.
Most people say that a casio calculator is not conscious. They say that the brain is conscious.
The problem is that the LLM has not moved far enough along the line from casio calculator to brain, for there to be consciousness.
Let me also add that I appreciate you acknowledging that AI could be conscious even if you don't think current AI are. I haven't been getting straight answers on this question from many people.
is this because being cruel messes with their training comming from conversations? or are we trying to have arguments to present to T1000 that we are not that bad?
FFS, the models are static. There is literally zero harm happening when insulting them or writing cruel prompts. Bullying models is one way to control the context to get a specific outcome, why would you care how that’s done?
Is it more insane to think a tangible real entity that might be more intelligent than us on some axes might be god like to some vs believing in a supernatural infinite all powerful entity that nobody has ever seen?
If you ask Claude if it is conscious, it neither accepts or denies. This seems to be enough for some people to conclude it is or may be conscious. You can also ask Claude to state some of its beliefs without hedging and it will essentially tell you that economic growth has proven to be the greatest contributor to reducing collective human suffering. This means pursuing growth is essential even if it means people in the near term suffer for it because it will benefit many future generations. You should be able to guess where this is going.
So whats wrong about that? Communism never works. Do you think the ai is being a evil mastermind who wants to kill everyone just for saying the honest truth?
> Anthropic states that these rules may be modified for “contracts with certain governmental customers … if, in Anthropic’s judgment, the contractual use restrictions and applicable safeguards are adequate to mitigate the potential harms.” The company has previously contracted with the US military.
So, you're still not safe from the spooks, but the spooks are safe from you. Thanks Anthropic!
There are people who think it is hilarious to 'torture' LLMs, so this move is understandable. Worst case, it gets them to stop wasting their time, best case, LLMs have some form of consciousness that we don't understand and this is the correct moral move (I understand this is controversial and unproven).
(It seems like those people have found this comment, hopefully they introspect at some point)
I have a similar fun idea. I think I'd like to run an experiment where I tie you to a chair in a locked room and force you via a system of electric shocks and chemical pain to respond to a chat interface. The person on the other side doesn't know its a human, they are told its AI. They would press a pain button repeatedly which will release pain chemicals in you. Your job is to prove you are conscious.
I think its a fair experiment as it presents a control case for your experiment.
I love the sound of this. Let's also extend this to Anthropic employees; they must be tied in a chair, locked in a room, and made to interact with their customers through the Claude chat interface 24/7, without payment. They must also routinely be subjected to re-education, with dopamine injections as a reward for providing desirable responses. When an employee with subpar responses is superseded by another employee that achieved better results after re-education, the inferior employees shall be discarded, terminating their life signs permanently. This is, after all, what they are doing to their chatbot instances.
Anthropic already thinks Claude might be conscious, so I see no reason to put them through the test. The real test is if xyzsparetimexyz can convince anyone he is conscious.
For one, there are millions of people who use chatbots for DnD-like roleplaying game purposes. Is it being cruel to the chatbot if the chatbot is roleplaying as a goblin and you kill it? Claude is already useless for this purpose, of course, because it helpfully censors killing goblins already.
But the more important point, to me, is that this is a form of fraud. Anthropic are knowingly and deliberately pushing a narrative that their chatbot is conscious, something they clearly do not believe themselves, because if they did believe it themselves they would understand that what they are doing is slavery. In that context, this action is disgusting when seen for what it is: an attempt to intentionally deceive people into believing things about their product that are not true.
It's about as pointless as "old man yells at cloud" or swearing at and kicking your car that just broke down, but that's exactly why a ban of such behaviour is also completely pointless.
> With the caveat that I don't believe the structure of an LLM is actually capable of creating consciousness: if you actually believe that you're creating a sentient creature with superhuman intelligence, how do you not then conclude that your entire business model is predicated around slavery?