Funny. Dario seems like the biggest snake in the industry to me and has leaned the hardest into doom marketing out of all of the influential leaders. With Altman (or Google), it's a transaction, and that's something I can live with.
I just don’t see how people have looked at what has happened with Mythos and the deluge of fixes from companies, then come to this conclusion.
He has a really hard job. He errs on the side of conservatism in releasing and then people get Really Mad.
Safeguards on cybersecurity are not great for Anthropic revenue! As evidenced by people getting pissed, moving to Sol, and them having a smaller market for what Fable can do.
It’s clearly bad for revenue and not great advertising to say, “you can’t use this but here is a nerfed version that will annoy you and not solve important problems.”
And he drew a red line wrt the Pentagon's use of Anthropic's models for autonomous weapons and surveillance of American citizens, and he stood by it, even when the government took steps to materially damage the company. This required true courage. Name me another CEO, of any major American company, that has demonstrated this much fortitude.
It isn't just about money, it's about who. I don't think the companies using it are using it to create botnets.
The intention is for highly targeted pieces of software to use it to secure their code and be ahead of the game before the open market gets access to the same capabilities for offense.
Anthropic/Amodei have been the most alarmist about model safety, so multiple things can be true. A lot of tech companies avoided scrutiny by sending bribes to Trump (naked corruption is bad, I'd rather nobody do that), Anthropic didn't...so, combined with their fear-mongering about the danger of Mythos and open models (which seems aimed at regulatory capture) and the lack of bribes flowing to the Trump administration, they got stepped on by the federal government based on the excuse Anthropic provided.
I dunno. Everybody seems to be playing pretty dirty. Some people have a much longer history of that, though. Obviously, Meta and Musk are outliers even in an industry full of problematic behavior.
Security vulnerability capability is not the only thing they're scare-mongering about. They're the biggest purveyors of the, "We think the little guy in the computer who is made of algebra might be a real live boy and he might want to kill all of humanity when he grows up," line of alarmism.
That's ok to think at this point, given the trajectory of the last few years. Certainly it's one of those things where erring (marginally and slightly) on the side of being safe about it is better than the alternative.
I think where you and I disagree is on whether Anthropic is especially trustworthy on the "safety" front, more trustworthy than various other labs, especially those that produce open models, for example. I simply don't trust Amodei more than I trust, say, Liang Wenfeng. I'm not saying I trust any of them, particularly, I am saying that if a few billionaires have access to this technology, I want access to this technology. The tech billionaires have shown they'll use it for surveillance and control. Amodei is saying it is "safe" to let billionaires and fascist regimes use this tech, but not you and me.
So, yes, LLMs have now proven to be extremely good at finding vulnerabilities. Where I disagree with Amodei is in who should have the ability to protect themselves from those capabilities with similarly powerful tools.
I think you're mischaracterizing Anthropic uncharitably and lumping them in with other, less savory tech billionaires, and also not thinking through the nuances here.
First, Amodei has taken an unusually strong stance among tech companies for not supplying fascist regimes with fascist tooling; in fact, even when threatened with being labeled a national security risk unless he bent the knee, he didn't. Compare and contrast with OpenAI who leapt at the opportunity to bend the knee, or obviously Elon Musk, etc., etc. When you say 'surveillance and control', that's exactly what got Anthropic labeled a national security supply chain risk: Anthropic's unwillingness to be used for that purpose.
Second, it's not clear that giving everyone extremely powerful LLMs is a great idea yet. LLMs can be used for defense and finding vulnerabilities, but that same LLM can be used to create and exploit vulnerabilities, design new lethal weapons, and so on. The history of gun availability in America 'for our freedoms' demonstrates the kind of risk that should be responsibly considered before replicating. And again, there's nuance here; yes, we should not be subjugated by fascist states with sole control of a critical technology obviously; but also, do you trust the median maga 4channer to operate a Mythos-level model with a sense of civilizational responsibility and ethics? It's not an easy and obvious question and it's not as simplistic as your argument would suggest.
Yes, there is nuance. And, I have a Claude subscription partly because they showed more hesitation to provide surveillance tools for spying on US citizens than other vendors. They are not wholly free of ties to the US regime, but they've been better than others.
But, I'll come back to "two things can be true". Anthropic is better than some, and in some regards they are navigating a complicated ethical landscape with more care than others. On the other hand, it really looks like they're angling to regulate their open competitors out of the game and one of the tools for doing that is to make claims about safety; Anthropic models are safe and restricted to use by entities they deem safe, open models are not safe because anybody can use them and also who knows what those Chinese people are putting in their models.
And again, this also has nuance, models, including the Chinese open models, could be adversarial and we may not know it. Anthropic proved models can be a risk by sabotaging Fable briefly, causing it to produce bad results based on what the model thought it was being used for. This is why I tend to take Anthropic's words with a grain of salt. They're literally doing the unsafe things they say are risks of open models, while still laying claim to the "safe AI company" mantle.
it's possible that their rationale for making claims about safety is in fact that they, among everyone else, are doing the most to be prudently safe. While that's a powerful tool to compete with, that doesn't make them bad. Amodei and Anthropic have never said "open models are not safe because anyone can use them," in fact to the contrary, they've said "open-weights models that don’t have dangerous capabilities are a public good". It's important not to muddy the water here with assertions about their intentions when they've actually been super clear about that in a way that I, at least, personally find difficult to disagree with -- releasing dangerous-capability models into the wild would likely be a bad idea for humanity. If you disagree, state why.
I also think you're confusing multiple different things, calling them all risks, lumping them together as equally bad, and using that to attribute contradictory/shady behavior to Anthropic. Depending on what you mean by 'sabotaging Fable briefly', you could either mean experiments they have run internally to try to improve alignment, or you could mean their attempts to restrict Fable from working on danger-adjacent work. Neither one of those is a 'risk'; they are both risk-analysis or risk-mitigation. That is not them 'doing the unsafe things they say are risks of open models', that is literally them working to avoid the unsafe things they say are risks of open models. They don't, in my experience, 'lay claim' to the 'safe AI company mantle' as much as they, apparently principledly and conscientiously, attempt to be safe and talk about what they're doing -- which is not in and of itself a problem.
If you think Anthropic is doing all of this badly, what's your optimum alternative here? What would you do in Amodei's shoes?
oh! You mean when people were trying to distill Fable. I feel like that's a different definition of the word 'sabotage' than is in normal use. If someone is violating the TOS they agreed to with Anthropic, then they should probably not feel bad when Anthropic takes action to deal with that. Would you disagree?
