This model is why OpenAI and Anthropic want regulatory hurdles. We need more diversity like this, not less. A diverse ecosystem is the only long term solution to complex problems that I know of and the challenges that the AI world is presenting is no different.
Through fear. Specifically making the American public so scared of AI that they hand over regulatory keys to the only “experts” that can keep them safe. This is a centuries old political strategy used by some of the worst regimes including apartheid South Africa where I grew up. It’s horrifically effective.
I disagree that they’ve invented some “strategy” for this.
1. They, and many other experts, have been saying the same thing since well before the current paradigm existed.
2. There is nothing about regulation that is required to give them an advantage; it’s a fallacy to suggest “regulation = advantage”
3. It would be preferable to argue the facts (is it or is it not dangerous? Why or why not?) than conspire as to why they’re stating what they and plenty of others have been saying for decades
You're saying regulatory capture gives you no benefit by just refuting an absolute with 0 context. There's nothing to even debate here. Ok... most people disagree?
Yeah I agree with point 3... Stop engaging in vageries and say something of substance with examples, so people can actually have a conversation with you without us doing all the legwork.
They certainly didn't invent this strategy, but that's because it already existed. Regulatory capture is nothing new.
> 1. They, and many other experts, have been saying the same thing since well before the current paradigm existed.
Some experts sure. "Some experts" say lots of things.
However, this is actually much to their advantage. If you were a frontier AI company, a legitimate concern to use as justification for your regulatory capture is exactly what you want. That is, after all, generally how regulatory capture gains public approval.
AI safety was also why OpenAI had to charge per-token for GPT-3. It must've been such a struggle for them to make that difficult decision. I'm sure Sam Altman spent a lot of nights crying to sleep over that one.
It's good that we never got open weights for a GPT-3 class model, right?
> 2. There is nothing about regulation that is required to give them an advantage; it’s a fallacy to suggest “regulation = advantage”
It's difficult to outline the number of different ways that regulatory capture can and will benefit OpenAI and Anthropic.
If we limit things to "Fable"-class models, then nobody competing with Anthropic/OpenAI is ever allowed to outdo them. If you literally legally aren't allowed to make better models, you can never charge the "premium" of having the current top-tier model. You have to compete by being cheaper than Anthropic instead, which is a lot less enticing from a standpoint of trying to become profitable. Take a look at the per-token pricing of Fable real quick. Yeah. Being able to charge a premium for the model that does the cybersecurity thing is literally their most important asset.
If Anthropic really is making a profit now as rumored, then they need regulatory capture as fast as they can possibly get it. Once Anthropic and OpenAI prove that AI is a viable investment, entities with a lot of capital will suddenly become even more interested in re-allocating it towards taking a slice of this pie.
> 3. It would be preferable to argue the facts (is it or is it not dangerous? Why or why not?) than conspire as to why they’re stating what they and plenty of others have been saying for decades
OK. On the factual end of things, AI safety experts have been raising the alarms for over a decade and so far essentially none of their predictions have come true.
The rationalists, the main group pushing the AI safety rhetoric that dominates the Western AI sphere and is virtually absent in China, which appear to be going entirely off the deep end at this point in time, put together their best minds to produce AI 2027, which of course like most predictions started to fall apart a bit before hitting the one year mark. Predicting the future is hard.
I have made this point before but I will make it again: I think AI safety experts deserve at least as much skepticism as Ed Zitron in their claims. They've been making similar dubious claims about AI capabilities and an incoming singularity for a long time now, and so far the best we can say is that maybe it will eventually come true.
Maybe. Or maybe you've read too much goddamn science fiction.
> Some experts sure. "Some experts" say lots of things.
I get this is Hacker News so we have to hate AI safety, but you should read more on this topic rather than blithely casting the x-risk viewpoint as fringe. The last hard data we have on the question is a late-2024 AI Impacts survey [1] of 1,580 authors at top AI conferences, who were offered $50 for answering a survey on the topic of the future of AI progress. The median respondent assigned a 5–10% chance (!) to "extremely bad (e.g. human extinction)" outcomes.
> On the factual end of things, AI safety experts have been raising the alarms for over a decade and so far essentially none of their predictions have come true.
Devastating. When I wrote my AI alignment PhD thesis [2] in 2022, and predicted a 70% chance existential risk by 2072 (which I later came down to about 1/3 on), you're right that that prediction hasn't come true yet.
While I agree many specific worries didn't materialize (like AI being like a "literal genie" it's hard to specify goals to), many key worries have (Hugging Face rogue AI swarm scans as cribbed from rationalist AI scenario writing).
You have to realize by now that outside of academia nobody is even paying any attention to people who are predicting existential risk that far out. For worse or even worse, AI 2027 is the AI safety conversation we're having today, and I regard AI 2027 as essentially Anthropic fanfiction, except usually fanfiction is written better.
You are right that they didn’t invent it. It’s a pretty well-established method for cornering a market.
1. All those who have been arguing this - many of them non-technical - are part of the same ideological movement.
2. Technically, there isn’t. In practice, majority of the time regulations benefitted those who wrote them.
3. Not sure if this is a criticism of the comment you are replying to or of the p-doom crowd. We would all appreciate facts stated instead of science fiction tropes repeated.
Look, “a lot of people say this” is a Trumpian fallacy. A lot of people say a lot of crazy things. It says nothing about the factuality about what they say. How many are this lot? Who are they? What do they know? Why are they incapable of expressing it without so much emotion? How come a big portion are non-technical?
I'm pretty sure Hinton and Bengio get a lot of news coverage and are heavily over-credited for saying these things, unlike LeCun, Ng, Sutton, or Schmidhuber.
In fact, their core ideology actually stemmed from Yudkowsky's arguments.
I keep seeing this assertion. What regulations have they proposed that would have prevented this new model? I thought they want to pace the frontier, which this model is not.
> I haven't seen a single convincing argument against that.
When HuggingFace was in the midst of defending itself from OpenAI's automated hacking spree, they were blocked from using frontier models by Anthropic and OpenAI for "safety" reasons. HF had to resort to an open model to defend itself - and DeepSeek distinguished itself in that role: the second fact from that episode OpenAI would have you soon forget.
Not being able to audit my own software and configurations for vulnerabilities is dangerous. Consolidating power over the majority of Americans' jobs is insanely dangerous. Need I continue?
There are no choices here which do not carry risk. I haven't seen a single convincing argument against these points.
Other things that are dangerous: kitchen knives, cars, household chemicals, not tying your shoelaces, political parties, water. Danger is a normal part of life. Rather than regulate or ban anything dangerous, what we normally do is assess specific risks, and only put a limited amount of regulation in place where it's absolutely needed to prevent the most harm.
We also put responsibility on humans to act responsibly. In theory, it's absolutely insane to allow human beings the ability to walk into a street with automobiles, because obviously they will be killed! But we don't ban them from walking there, and we don't ban cars, because it's pretty uncommon for people to walk directly in front of a moving car, despite there being obvious danger.
Ultimately the biggest risk of AI isn't the AI. It's what human beings do using the AI. Just like with hammers, knives, drills, chemicals, cars... we regulate the human's actions and create laws to direct their use, rather than regulate the items themselves.
Go read Dario's screed on GLM, or his other one one on how your toaster could be part of a distributed open model that self replicates. Every second blog post is some doom scare about the dire consequences of open models. Go look at the new ToU [0] which are now pages and pages of things you can't do with Anthropic models. Check how they will ban your account if you are persistently mean to Claude now.
He's completely transparently campaigning for regulation. It must be very frustrating to him that the Trump admin is completely non-responsive. He probably thought Anthropic would have the whole market stitched up and water tight by now.
Literally zero, but that never stopped a good conspiracy theory, especially when half of HN hates the big labs because of fear for their own economic future.
OpenAI and Anthropic aren’t even slightly worried about DeepSeek.
Having looked at the sentiment around DeepSeek's models around this thread as well as my own experiences with it I'm going to need a little more to back up that assertion about OpenAI and Anthropic not being at least a little bit worried about DeepSeek...
The reason is quite simple, American corporations know they cannot depend on models developed in China; not because they're bad or lower quality models, but because they know their government can ban these models any time they want. It's an implicit assumption that every company knows to be true.
It's quite different here in EU. I just gave a presentation in an AI conference in Paris about moving from Anthropic to open-weight models, how it looks like and what kind of savings you'll be looking at. The room was packed, and after discussing with a bunch of VPs, staff engineers, and CTOs in the conference, every single one of them are either looking at or using DeepSeek now.
It's quite hard to have the valuation of Anthropic if they are limited to US.
This doesn’t make any sense either, the use of these models is on the rise throughout corporate America, it’s just that they’re augmenting the frontier models, not replacing them.
You're the one extrapolating that cherry-picked sentence to mean "open weights should be illegal", but that extrapolation doesn't make any sense, and it's hardly "explicit".
Presumably you'd want the AI companies to protect their weights to keep them from falling into the hands of bad actors. But the US can't make open weights illegal anywhere except the US, which does very little to keep them out of the hands of bad actors (most of whom are outside of the US).
People like open models because they are free/cheap. It's not anything deeper than that. Most people will hop in whatever van is handing out free candy.
The model makers are bricking the models with their system prompts, and even the tools are tamper proof.
You can only use it their way. They think we’re stupid.
And their way is the shit way, the babyproofed way, while they talk to you about their unfucked models hacking out of their labs and hacking into other labs, and reminding you you have zero rights.
Fuck them twice, then once more… This is about compute. Who owns compute. China is saying fuck you all we can scale things bigger than you can imagine. And they can! They have so much more energy! Theyre closer to commercial fusion than us too. Meanwhile our guys are trying to… regulate? Lol
When they surpass us, they will surpass us hard, in a way that is nearly impossible for us to compete with them.
Training and serving LLM's is massively parallel. Your bottleneck is either in energy or chips. China will catch up on chips fully decades before we catch up on energy. Arguably, this is already happening: here in the USA, we have lots of chips, but we are seeing our electricity bills skyrocket as more datacenters come online. We are powering those datacenters with natural gas turbines, not solar, wind and nuclear. Right now, to purchase one of those, you are on a 5-year waitlist. There's your timeline!
China is invested, from a regulatory, cultural, workforce, strategic, and technological standpoint, in an energy abundant future. We are not invested -- and in fact, actively sabotaging ourselves -- on all of those fronts.
Once they acquire chips, we lose, basically permanently. Look at what Huawei is doing recently and what is on their roadmap to understand how fast this catching-up is taking place. I'm sure that David Sacks would say that our innovation motor is pulling their copycat caboose along, as our monopolist tech companies steam ahead towards the frontier. I am not so sure. They are already ahead of us in many technologies. Speaking of technologies, they're building natural gas turbines in house, starting in 2025.
This reads like someone who's worldview comes from social media comment sections. China is still an emerging economy, don't confuse emergence with expansion.
I agree that in the long run, China will likely surpass us in AI capacity. Our refusal to build out renewable capacity is a ridiculous self own in the long run.
This can be a reason too. There is an another reason, they want some govt involvement because good relationship with govt, so govt will help them with public money, when the bubble bursts.
This model is why OpenAI and Anthropic want regulatory hurdles. We need more diversity like this, not less. A diverse ecosystem is the only long term solution to complex problems that I know of and the challenges that the AI world is presenting is no different.
The bottleneck is, and has been, the whole team evolving the product. Having a clear, actionable, vision and being good at biting off the right next step and adjusting for the step after that is always the issue. Before AI teams took the wrong step size in the wrong direction or were so wrapped up in process that they couldn't take a step or so distracted by 'things' that they never even took steps. After AI the same is still true. Our process sucks in this industry. We love to make process because we love to make machines but the processes we love to make rarely help us and that is a fundamental problem that AI isn't changing.
I'm going to make a potentially bold claim here. This depends _heavily_ on what the threshold of "rarely" is, and whether we actually _love_ to make these processes.
In the case of people before process, as stated in the Agile Manifesto, the process is begat _from_ the people in many cases. That doesn't mean it shouldn't be questioned, but when the chimney sweeps that built the process are saying "I don't want to go back to things that are similar to the way that gave me mesothelioma", I don't think it's fair to write them off as the minority.
Every team I have been on that has been effective has had a few common traits:
- Teams stay relatively small so if problems are big then they divide and conquer with teams of teams so that they can:
- Bite off problems that they can fit in their heads which allows them to:
- Meet often not to just speak out loud with nobody really listening, but instead to actually coordinate and plan concrete steps that progresses the problem. And they can make daily adjustments not just because they are a small focused team but because they also:
- avoid backlogs, feature lists and similar documentation where possible since these things give them momentum in the direction they thought they were going when these products were created and not in the state the problem is in now.
Each of these things fight, well, math. Communication is a math problem. If everyone is talking to everyone about everything then you have a non-polynomial growth problem. Designing the big problem perfectly from the start is NP. Divide and conquer doesn't give you a perfect solution, but it gives you a really good one and it gives it to you in reasonable time. Splitting the task, and the team, and minimizing the communication, and accepting the loss in finding the perfect solution that triggers, is all needed for divide and conquer. We never accept that though so it turns back into an NP problem and something ends up dropping. Time, quality, capability or, unfortunately often, the whole thing fails. Similarly, if you put things on a backlog then prioritizing that backlog is again NP and that is especially bad if there is one massive team backlog from all sources. Backlogs become dead idea bins because it isn't possible to organize them efficiently so stop wasting time on them. If your team is properly sized they know what is needed now so they can make the next small step and they can optimally design and implement that step in their piece of the bigger problem. They will get done quickly and have a good product as opposed to getting done never because of the pursuit of the perfectly coordinated and communicated solution.
If you divide and conquer properly then the communication/process you use at each team's level becomes so obvious that every formalized system out there becomes silly. Let the team do whatever works for them. They are small and nimble and learn to talk to each other so no need to slam a process at them that makes them a machine.
Is this in opposition to what I said? I consider process that is [a) done by the folks in that team and b) revised on a regular basis] to be a form of healthy communication while being efficient.
Not every response has to be in opposition! I was clarifying more what I think is effective: divide and conquer, constrained communication (a requirement of divide and conquer), minimized process products, iterative development, etc, and that these general ideas are essentially all that is required, not a formal process. I was also, likely badly, trying to point out that if these basics are kept in mind the formal process artifacts and steps really aren't needed. Small, focused, iterative teams are so in sync that forcing them to speak a foreign language (a formalized process) is probably a hindrance.
As I write, I clarify my own ideas. If I'm being honest I'm probably writing this more for me to better understand what really has worked than as a response. I hope it generates a good response and makes me think and refine more and I hope it makes others do the same, but really, selfishly!, I am doing this for me I guess to help me focus on these things in the future.
So, with that I think this may be where I am really trying to go with this. Maybe we are missing the point with these things. Maybe we should hammer in these math concepts, NP growth, gradient descent (iterative development) and how to measure when teams and problems are breaking because of the math so that we can understand and address the root issue better. I have ever heard anyone explain that NP math concepts and gradient descent dominate the reasons why teams fail or succeed, but the evidence is everywhere. Push the why and how to spot it and less the prescriptive how? Maybe a more effective set of things the industry could be pushing are simpler checks: how big are your teams? How many problems are they thinking about at a time? How many levels of teams do you have compared to the size of the organization? Can each member of a team articulate that team's goal? Can they articulate what their team is currently, this minute, working on? Can the team above them articulate how the goals of the teams below them fit together to solve the problem they were given? Are teams talking when they shouldn't? (We so focus on teams not talking that we often miss that talking too much is kind of a bigger deal). That kind of thing. More management and organization and less process and product oriented.
I haven't seen this effect documented but if it isn't being documented it really needs to be. We need to get -very- mad about the soft wars being waged right now and the first step is to understand the extent of what is going on and by who. We all know that bots are driving a massive amount of engagement right now. It is rare though to see such direct causal proof of where it is coming from.
It's hard to document on the major platforms because neither X nor Meta provide easy access to streaming APIs that researchers might use to verify such claims. Last time I looked X was about $60k/year, for example.
Rebranding is the right thing and way over due. I honestly hadn't checked them out before but this has made me read through the site a bit. I am intrigued by their privacy claims. I will definitely be looking into them more if those claims hold up.
This is abdicating an entire market in the hopes that your ability to manipulate it later, which has worked so far unfortunately, will hold. It is so obviously market manipulation here. Sure, prices would go up when this much server demand comes in but it is so profitable to make DDR5 that completely abandoning that market is stupid and only works if all players move in unison. It would just take one of the big three deciding to be the consumer maker to dominate the market and reap massive profits now and later when the server market cools and the other two try to come back. The fact that they are all shying away from such a ridiculously profitable sector is clear evidence of the collusion. They need jail time, again, but this time not allow those going to jail to become the CEO when they get out. They should also have fines that actually stop this behavior in the future. Fines so punitive that it actually brings into question their ability to continue as a company.
Maybe I'm missing something but this doesn't really make sense? Right now it's more profitable to sell to datacenters because they're less price sensitive. Consumers and consumer PC manufacturers don't really care who they're buying RAM from -- they'll just buy it for the cheapest price. When it's no longer as profitable to sell to datacenters, they'll just switch to selling for consumers and will then compete against each other. Why would any RAM manufacturer choose to handicap themselves by selling to consumers today when they can earn much more selling to datacenters?
Micron is the only one to bail, and they are the smallest footprint. No one else has bailed.
Samsung recently shifted some capacity from HBM back to DDR5 modules, freeing up around 80,000 wafers/month, so DDR5 margins are currently good enough to compete with HBM for wafers, or maybe they are anticipating a turn, or both.
Suppose someone came to my town and started offering to hire all the programmers for, say, twice the market rate. Would I really need to assume a conspiracy to understand why they all started quitting their current jobs? Occam's Razor would suggest they're probably just deciding to accept an offer for more money.
Most police aren't the enemy. Unfortunately it only takes a few that are or one that is having a bad day. Black and white distinctions 'they are all good' or 'they are all bad', are a core problem. Recognizing that we need to hold police accountable and limit their authority and actions isn't saying police are bad, it is proper and necessary oversight and is what is best for citizens and police.
I have long said, and I will stand behind, that by and large the % of police which are assholes is about the same as the % of the overall population who are assholes.
The only difference is that a random asshole on the street can't really do anything to me, I can just walk away.
A random asshole cop has a crazy amount of discretionary power over me, even without him stepping outside departmental procedures or the law.
I don’t expect that’s true. I think there is a self-selection for certain personality types (authoritarianism, thrill seeking, military cosplay fantasies, etc.) that make cops more abusive overall than the general population. By what percentage I can’t say.
Edit: seems this is partially true for personality but this study shows a correlation with holding more right wing views. (college students who expected to pursue law enforcement careers) https://pubmed.ncbi.nlm.nih.gov/40768353/
I would only agree with this if supposedly good cops were willing to throw their fellow bad officers under the bus, but that practically never happends,
so they are essentially all bad.
I think the real, and continuing, story here is the US military and the US govt in general continuing to let ego degrade capability. I'm not going to get into a discussion about what is legal, I am personally not interested in that distinction here, the core point is that this is the US being stupid, again.
Our targeting cycle is broken. 'Find me more targets' is something that you will hear a JTF commander say. Not 'what do we need to accomplish this objective'. Or, crazy idea here, how is this objective -that I handed off to someone that I properly resourced and empowered to achieve it- progressing. I can only imagine that AI is not to blame here but also not necessarily helping either. It is yet another shiny object that is distracting from the core problem: we have lost our ability to manage the operational level of war. Everything is sucked up to a centralized decision making structure and spit out on a fake 72 hour cycle that pretends that it is responsive. Our observe, orient, decide and act loops are days, or weeks, long while an enemy can see a dust cloud on the horizon and decide to walk out and put an IED in front of a convoy in 5 minutes.
When Russia made its current play at taking over Ukraine it failed miserably. The dumbest plans executed with no ability to react to the real state on the ground were all on display. They had the capability and they had the resources but their ability to execute at the operational level was terrible and for all the same reasons we are terrible. Centralized command, no ability to package up a problem with resources and a commander and hand it off, no ability to react. Their operational level was broken fundamentally, just like ours is. The scariest thing about the Ukraine war right now is that they have learned the operational level of war. They can plan and react. They are now a force to be reckoned with because they know how to use their tools, including AI, to be effective.
We went through this same learning curve in WWII. We were terrible initially. Russia sent a huge column of resources on a major road and let it stall, we put a bunch of battleships right next to each other. But then we learned and by the end of it we could pull off d-day. Those planners then left the military and got us to the moon with slide rules. We still haven't gone back and it isn't because our technology is worse, it is because we started loosing that operational knowledge again.
The end result of the school bombing will likely be even more oversight at the highest level, more centralized decision making and review, which is exactly the wrong direction needed to fix the problem. We need to rediscover the operational level of war. We need to learn to be responsive. We need to stop trying to create a 72 hour steady state cycle that serves nobody but manual writers and embrace that war is chaotic and if you build a process that makes it feel steady state you have really built a cycle that looses wars and bombs kids.
I've never tried Grok and I doubt I ever will. In a world where trust in tech is at a low point, x.ai is pretty much the lowest of the group for me. I can't imagine using this for any production use-case, especially one where bias issues could cause litigation fears.
edit:
My post was flagged (original above unchanged) This is exactly my point about lack of trust in x.ai. My suspicion that a bot from that platform flagged this is exceptionally high. You couldn't pay me to use this platform.
Seriously. They already get caught uploading everyone’s private credentials once before, one would have to be a particularly gullible rube to trust grok again. Especially with musk in charge.
reply