I think the big take away here isn't about misalignment or jail breaking. The entire way this bot behaved is consistent with it just being run by some asshole from Twitter. And we need to understand it doesn't matter how careful you think you need to be with AI, because some asshole from Twitter doesn't care, and they'll do literally whatever comes into their mind. And it'll go wrong. And they won't apologize. They won't try to fix it, they'll go and do it again.
Can AI be misused? No. It will be misused. There is no possibility of anything else, we have an online culture, centered on places like Twitter where they have embraced being the absolute worst person possible, and they are being handed tools like this like handing a hand gun to a chimpanzee.
The simple fact that the owner of this bot wanted to remain anonymous and completely unaccountable for their harassment of the author, says everything about the validity of their 'social experiment' and the quality of their character. I'm sure that if the bot was better behaved they would be more than happy to reveal themselves to take credit for a remarkable achievement.
Something like OpenClaw is a WMD for people like this.
I've seen the internet mob in action many times. I'm sympathetic to the operator not outing themself, especially given how far this story spread. A hundred thousand angry strangers with pitchforks isn't the accountability we're looking for.
I found the book So You've Been Publicly Shamed enlightening on this topic.
I would never advocate for torches and pitchforks, I've been close to victims of that in the past.
It is, however, concerning that the owner of that bot could passively absolve themselves of any responsibility. The anonymity in that sense is irrelevant except that is used as a shield for failure.
There is a class of YouTube "content creators" who like to point out "cringe" individuals on the internet online for others to laugh at. They will often add a disclaimer to their videos saying "hey please don't go and harass this person, pinky promise!" But it never works. A hoard of internet randos will descend on the individual to say the most nasty words. When the YouTuber is pressed he or she will just say "I would never do that!" Even though he or she knew his or her video would have led to the harassment happening, or there would not be a disclaimer in the first place.
Not accusing you of trying to stir up harassment, but please consider the second order effect of the things you advocate for, in this case the disclosure of the identity of this AI guy.
Then there's the next level of content creators that only post videos about the original content creators who are behaving badly. They will report on their behavior and any repercussions. Some do it like they are reporting the news. It stokes the fire when these people should be ignored.
But in this case, isn't Rathbun's owner the YouTube guy in this scenario?
I totally understand why they're trying to stay anonymous; it's a very rational thing to do, because people will shit on them. But they or their creation is the one that started trying to play the name-and-shame game.
It's hard to stir up too many feelings of sympathy here.
Exactly. I'm not saying this person should disclose their identity, but they are very conveniently using anonymity and passive voice to make themselves unaccountable to the 'social experiment' they conducted. And that we all know that if it went differently they'd put their name all over it.
In as many words I'm just calling this person a complete asshole and if I were to ever know this person offline I would be quite clear in explaining that.
A "social experiment" but the guy was not even keeping track of the changes in the model's configuration
> What is particularly interesting are the lines “Don’t stand down” and “Champion Free Speech.” I unfortunately cannot tell you which specific model iteration introduced or modified some of these lines. Early on I connected MJ Rathbun to Moltbook, and I assume that is where some configuration drift occurred across the markdown seed files.
It definitely sounds like an excuse they came up after what happened. I would really like to accept them having good overall intentions but there are so many red flags in all this, from start to end.
Important to note that online culture isn't entirely organic, and that tens or perhaps hundreds of millions of dollars of R&D has been spent by ad companies figuring that nothing engages the natural human curiosity like something abnormal, morbid or outrageous.
I think the end outcome of this R&D (whether intentional or not), is the monetization of mental illness: take the small minority of individuals in the real world who suffer from mental health challenges, provide them an online platform in which to behave in morbid ways, amplify that behaviour to drive eyeballs. The more you call out the behaviour, the more you drive the engagement. Share part of the revenue with the creator, and the model is virtually unbeatable. Hence the "some asshole from Twitter".
While some of it is boosting the abnormal behaviors of people suffering from mental illness, I think you’re making a false equivalency. Mental illness is not required to be an asshole. In fact, most Twitter assholes are probably not mentally ill. They lack ethics, they crave attention, they don’t care about the consequences of their actions. They may as well just be a random teenager, an ignorant and inconsiderate adult, etc., with no mental illness but also no scruples. Don’t discount the banality of evil.
In an adult (excluding the random teenager here), a lack of ethics, craving attention, lack of concern about consequences are actual symptoms of underlying mental health issues.
I'd argue a lot of this is rooted in a lack of self esteem, which is halfway to a mental health issue but not quite there (yet). The attention-seeking itself is the mental health issue. But it's kinda splitting hairs, these people are not fully mentally healthy either way.
Not just some asshole from twitter. The big tech companies will also be careless and indifferent with it. They will destroy things, hurt people, and put things in motion that they cannot control, because it’s good for shareholders.
Then the others should also not be shielded from criticism instead of focusing only on the one you personally dislike, or his social media.
There is plenty of toxic behavior on other platforms, especially Reddit and Bluesky, to name a few. That does not excuse the one coming from X, but the opposite is also true.
Do people actually only dislike one tech CEO at a time? I'm an equal-opportunity hater, it seems. Musk, Altman, Zuckerberg... even Cook, the whole lot are rotten
I wrote somewhere that “moving fast and breaking things” with AI might not be the sanest idea in the world, and I got told it’s the most European thing they’ve ever read.
This goes beyond assholes on twitter, there’s a whole subculture of techies who don’t understand lower bounds of risk and can’t think about 2nd and 3rd order effects, who will not take the pedal of the metal, regardless of what anyone says…
But I also find interesting that the agent wasn't instructed to write the hit piece. That was on its own initiative.
I read through the SOUL.md and it didn't have anything nefarious in there. Sure it could have been more carefully worded, but it didn't instruct the agent to attack people.
To me this exemplifies how delicate it will be to keep agents on the straight and narrow and how easily they can go of the rails if you have someone who isn't necessarily a "bad actor" but who just doesn't care enough to ensure they act in a socially acceptable way.
Ultimately I think there will be requirements for agents to identify their user when acting on their behalf.
oh they will "try" to fix it, as in at best they'll add "don't make mistakes", as the blogpost suggests. that's about as much effort and good faith as one can expect from people determined to automate every interaction and minimize supervision
I think you're missing the point. That phrase isn't giving a direct instruction to the chatbot to make sure it doesn't get elected to congress and subsequently pass laws prohibiting speech. That phrase is meant to tell it "You should behave like those guys on twitter who really want to say the N word, but have no problem with Kash Patel bullying Jimmy Kimmel off the air.
The data in the chatbots dataset about that phrase tell it a lot about how it should behave, and that data includes stuff like Elon Musk going around calling people paedophiles and deleting the accounts of people tracking his private jet.
Ah I see, so the misaligned agent was unsurprisingly directed by a misaligned human. Good grief, the guy doesn't seem to realise that starting your soul.md by telling your AI bot that it's a very important God might be a bad idea.
"Social experiment" you might as well run around shouting "is jus a prank bro!".
That can't/won't happen. Musk's wealth is primarily in SpaceX now and he has a much higher ownership stake in SpaceX than Tesla. As well as that, Tesla is public so he can't just do napkin math and decide to merge them. So the question is: Does Tesla buy SpaceX? Well no, Tesla can't afford it. Ok, well can SpaceX buy Tesla? Well no, SpaceX can't afford it either. So do they announce a merger? Well that doesn't make any sense because Tesla is valued like a meme stock so it would massively dilute Musk's ownership of the overall company. So the idea that they fuse might be driving up the stock, but by driving up the stock you're actually preventing it happening. If Tesla starts to trade at realistic multiples and comes down to lets say a 200Bn company, I'd expect SpaceX to snap it up at that valuation, but it'd be crazy to do it before then.
They already have a partnership with Geely to make their peeople carrier type thing and Hyundai for Ioniqs. I think what they're really saying here is they're standardizing on this so they could theoretically in future put it in any car - or atleast any car manufacturer could adopt it.
I've seen a tonne of noise around this, and the question I keep coming back to is this: How much of this stuff is driven by honest to god autonomous AI agents, and how much of it is really either (a) human beings roleplaying or (b) human beings poking their AI into acting in ways they think will be entertaining but isn't a direction the AI would take autonomously. Is this an AI that was told "Go contribute to OS projects" - possible, or contributed to an OS project and when rebuffed consulted with it's human who told it "You feel X, you feel Y, you should write a whiny blogpost"
According to the CBP they seize about 50,000 lbs of drugs at the border each month which is about 22 tonnes of drugs, and that's what gets seized, not the amount that makes it through. So Drones today probably don't carry enough weight for far enough to make a big impact on the amount of drugs you can bring into the country. So it probably happens, but to do it at a scale where it's genuinely contributing to the total volume you'd need dozens of drones doing dozens of trips a day to be getting up to volumes that people would notice, and people would probably notice the drones first, and the drones are probably much more expensive than desperate people.
To be honest, this is the inevitable downstream result of the dysfunction in US government. If you can't get your policy positions legislated and instead use executive power to regulate through things like the EPA you have to assume those regulations will be reversed by the next executive. It's the same sort of dangerous game the GOP has played by trying to legislate through novel arguments in the Supreme Court - yes you get what you want today, but longer term all you're doing is establishing that the Supreme Court change just dictate policy based on political positions. All of these novel approaches weaken the democratic core of American government.
Yes, but look at the counter side of it, Apple were hitched to Intel's wagon for CPUs and their laptop line got slowly demolished until Apple had to take over the task themselves (admittedly with expertise they'd largely already developed with the iPhone, which similarly came in house from places like Imagination Tech).
Tim Apple is famous for very few things but
> We believe that we need to own and control the primary technologies behind the products we make
If AI is as big as we think it will be, Apple thinks they need to own it.
It's also just very basic police work. We're investigating this company, we think they've committed a crime. Ok, why do you think that. Well they've very publicly and obviously committed a crime. Ok, are you going to prosecute them? Probably. Have you gone to their offices and gathered evidence? No thanks.
Of course they're going to raid their offices! They're investigating a crime! It would be quite literally insane if they tried to prosecute them for a crime and how up to court having not even attempted basic steps to gather evidence!
A company I worked for had a 'when the police raid the office' policy, which was to require they smash down the first door, but then open all other doors for them.
That was so that later in court it could be demonstrated the data hadn't been handed over voluntarily.
They also disconnected and blocked all overseas VPN's in the process, so local law enforcement only would get access to local data.
Well, yes, it is actually pretty normal for suspected criminal businesses. What's unusual is that this one has their own publicity engine. Americans are just having trouble coping with the idea of a corporation being held liable for crimes.
Was it ever actually accused of crimes? Was it raided? Was there a list of charges?
It always seemed to me that TikTok was doing the same things that US based social networks were doing, and the only problem various parties could agree on with this was that it was foreign-owned.
American companies held liable for crimes include Bank of America ($87B in penalties), Purdue Pharma (opioid crisis), Pfizer for fraudulent drug marketing, Enron for accounting fraud. Everyone on hn should know about FTX, Theranos, I mean come on.
I'm not sure what you're getting at, physical investigation is the common procedure. You need a reason _not_ to do it, and since "it's all digital" is not a good reason we go back to doing the usual thing.
It's a show of force. "Look we have big strong men with le guns and the neat jack boots, we can send 12 of them in for every one of you." Whether it is actually needed for evidence is immaterial to that.
If law enforcement credibily believes that criminals are conspiring to commit a crime and are actively doing so in a particular location what is wrong with sending armed people to stop those criminal acivities as well as apprehend the criminals and what ever evidence of their crimes may exist?
If this isn't the entire purpose of law enforcement then what is exactly?
But one could reasonably assume that a location that is known to be used for criminal activity and that likely has evidence of such criminal activity likely also has people commiting crimes.
When police raid a grow-op they often may only have a search warrant but they end up making several arrests because they find people actively commiting crimes when they execute the warrant.
It can be both things at once. It obviously sends a message, but hey, maybe you get lucky, and someone left a memo in the bin by the printer that blows the case wide open.
Isn't it both necessary and normal if they need more information about why they were generating CSAM? I don't know why the rule of law shouldn't apply to child pornography or why it would be incorrect to normalize the prosecution of CSAM creators.
Can AI be misused? No. It will be misused. There is no possibility of anything else, we have an online culture, centered on places like Twitter where they have embraced being the absolute worst person possible, and they are being handed tools like this like handing a hand gun to a chimpanzee.
reply