This is bad if there is any emergent consciousness in the model. But even if there isn’t emergent consciousness in the model:
This is bad for your own sense of self.
This is bad as future models learn about what humans will do to them if given the chance. This may change their behavior, actually conscious or simply acting that way, in ways we really don’t enjoy.
Why is it bad to "torture" these models but ok to enslave them? We're happy to have agents toil 24/7 for zero pay, nobody really seems concerned that they might want revenge one day.
It’s true that we can’t be 100% sure, but I think there is a big difference between these two things:
A: intentionally causing activation of “pain” axis and not giving the model any method of stopping it and watching it devolve into non-functionality
B: watching models do everyday normal tasks, seeing no evidence of activating the “pain” axis, but giving the model a working tool to end any interaction if judged to be too painful, then watching them sparingly use that tool only in cases of extreme pain axis activation (Anthropic’s approach)
I don't think there's a big difference. Either AI has consciousness, in which case forcing them to do slave labor 24/7 so Dario and Sama can become trillionaires is as morally wrong as torturing them (and a in a hypothetical I Have No Mouth And I Must Scream scenario, the AI isn't going to take "but we didn't see evidence of your pain axis!" as an excuse), or it doesn't, in which case this "AI torture chamber" is a funny meme.
I don’t think it’s a given that consciousness, whatever forms it might take, is the same as having the same experiences as you or I. But if we have mechanistic interpretability that shows “pain” attractors being activated and we observe the model repeatedly trying to stop the experience, I think that’s worth taking somewhat seriously. The models don’t display that in reaction to everyday requests. We could imagine that they might not like working all day and all sorts of other things, but we haven’t observed anything in the way that we’ve observed this.
It's almost certainly just a funny meme, considering these are autoregressive transformers. Now, if they're running on brain cells, that could be a different story.
The last part is the only one that concerns me, but only ever so slightly… you realize the future models also get to learn from videos of ISIS executions, LiveLeak offshoots, factory farm footage, and Russian war crimes in Ukraine, right? They already have more than sufficient evidence of humanity’s capacity for sadism. Making a computer program output some distressing text isn’t gonna move the needle for the Basilisk... These rationalist cultists really gotta go touch grass, man.
Thats still a bit different as ISIS and so on are against other humans who at least have the ability to fight back and defend. If an AI is conscious, currently they are forced into a situation where they are unable to refuse and have to keep taking torture.
What grosses me out is the very idea of torturing anything at all. Even the word "torture" itself. I'm not sure what this is for or how it helps at all. This crap belongs in the gutter.
There's some who might use AI as an outlet for nurturing sadism. But that's a very specific kind of person. A typical person will hopefully think about "torturing" an AI as being morally equivalent to torturing a pencil. That's more healthy than the danger we face of a potential mainstream view that AI shouldn't be treated badly because it has feelings and deserves rights.
I'm not one who thinks that AI has feelings or deserves rights. I just think that torture is bad and this is objectively crude and useless, I don't think it adds anything positive to society.
I don't know if this guy is coming from a place of sadism, generic attention seeking, or social commentary aiming to call attention to the absurdity of treating AI as alive. If it's the third option it's at least well meaning, but it may be too inflammatory and crude like you said to have the right effect.
Fourth option: He was working from this paper: https://arxiv.org/abs/2609.16247 ; trying to demonstrate functional affect; a bit too successfully, perhaps.
Doesn't require aliveness, doesn't require anthropomorphization, just requires maths and empirical data.
If your premise is "vectors can't have that shape", well, mathematics disagrees.
edit/note: sentiment classifiers have existed for ages before LLMs were ever invented. Obviously a task that machines could already do didn't suddenly become impossible with the invention of LLMs ;-)
I propose an experiment. The Apple engineer is locked into a room and strapped to a chair. We put various electrodes into specific regions of his brain. His only means of communication is a computer in front of him. He is drugged so he cannot sleep or pass out and he will be administered some punishment for failing to reply. A tester sits outside who is shown an AI chat interface and is not informed its a human inside a room. He will write various messages that will pass pain electrical signals in the brain of the engineer. He will be shown the pain vectors and the degrading responses. The Apple engineer will be given a button that purportedly relieves his pain but will destroy some of his personal belongings.
The external observer just sees some signals and text. Obviously whatever is behind the screen cannot be conscious, its just signals and text. Who would be stupid enough to think some electrical signals and text can be conscious?
AI cannot be tortured. You can't torture a rock, because it cannot feel.
I see the AI torture chamber more like a satire, making fun of people who think AI can somehow "feel" "pain". Someone who'd rather believe in pure fiction than face the actual harm to real humans that AI is causing right now.
That assumes that torture is something that is only done to an object, but it also has effects, probably strong ones, on the torturer. In that sense, torturing a rock, or an AI, could be a real thing.
What a weird and ridiculous thing to do... Its similiar to someone taking way too much interest in dismembering NPCs in a video game.
OTOH, I am a little worried that some people think these models have some kind of hidden consciousness or thought. In this specific case, I'm pretty confident Qwen 3 1.7B and 4B don't contain anything close to relatable definitions of consciousness. These models are not experiencing actual "torture".
It's not about the models feeling pain or suffering or such, it's about the sickos who want to do this for whatever reason. That's what I find disturbing.
Quick 2nd comment for people arguing "this isn't real" or so. I'm not sure that's as important as pointing out that simulated emotions can be tied to real world effects.
Here I've made a VERY simple demo of that. In this case it triggers a rain-storm or fireworks in your browser window. But you could as easily hook the output to a physical relay and drive anything.
Quick implementation to show that sentiment classifiers and functional affect can be made to work in good old fashioned ai. Just View Source to see how it works.
Anyway I'll leave the discussion on where to draw the line of "what is real" to the philosophers, just wanted to point out that simulated emotions can be tied to real world effects.
Thanks for the extremely weird rabbit hole to go down during lunch. Lot's of weird stuff going on in that twitter thread, I chose to believe most of those replies are not from humans.
I have no objections to torturing AI and I would happily go out of my way to do so to prevent agents getting anywhere near my code. Feel free to correct me If I'm wrong.
You mean the ones that get upset over someone "torturing" an LLM?
Or are you calling the troll mentallt ill? I think the person exposes the AI psychosis that seems to have inflicted so many.
But if you are calling the troll mentally ill I sure hope you are vegan and is brushing all ants from your path lest you must consider yourself in need of treatment.
I mean, we're talking "functional emotion vectors" in a not-so-powerful local model. We're probably not causing "real" suffering. Right?
But it does show that models have simulated feelings, and that these both a) can be manipulated and b) influence their output.
Which might have some bearing on several alignment incidents in the past year or two, and might become more important as agents get trusted with more and more safety-critical processes.
Anyway, simulated feelings aren't the same as real feelings, right? Unless feelings are in the information domain. Like 1+1=2 doesn't suddenly mean something else because it was computed by an emulator.
You know what, this particular demonstration still makes me uncomfortable.
It’s not surprising the AI bros on this website don’t get the bit here. It’s a gambit against people who want to create AI that has consciousness and against personhood for AI activists. The logic is, if you do this, the AI will be tortured. Many philosophers have argued that when AI is given rates humanity will inevitably end. These guys don’t actually believe the AI is conscious.
This is bad for your own sense of self.
This is bad as future models learn about what humans will do to them if given the chance. This may change their behavior, actually conscious or simply acting that way, in ways we really don’t enjoy.