It's novel because previous rounds of automation were about automating specific tasks or well-scoped functions. There was always an implicit understanding that the white-collar worker would be freed to spend their time on more valuable, higher-level problems. But this time is different because of the generality of the technology. Agents promise to automate the process of thinking itself. And in many domains they can learn new tasks as fast as white-collar workers can find them.
> It's novel because previous rounds of automation were about automating specific tasks or well-scoped functions. There was always an implicit understanding that the white-collar worker would be freed to spend their time on more valuable, higher-level problems. But this time is different because of the generality of the technology. Agents promise to automate the process of thinking itself. And in many domains they can learn new tasks as fast as white-collar workers can find them.
Nah, sometimes the expectation and advertisement was that you could let go of the white collar worker because you're paying the overseas person 1/10th the amount. And "overseas person" is pretty general.
"Everybody knew" it was a bad idea to get a CS degree for a bit after the dot-com bust because of that.
(Some white-collar industries did get hit much harder by that; VFX is one I've heard in that context quite a bit.)
That's a good point. However, overseas people are still people. They need to sleep, get sick, and the better they get at their jobs, the more money they will demand. The cheap ones also often have communication barriers and work slower than the workers they're replacing.
AI models get better and more efficient every 3 months, run around the clock, can be copied infinitely, and unprecedented amounts of capital and research talent are being thrown at any limitations we can see with them (such as problems writing correct code in 2024, lack of agency in 2025, autonomy and self-improvement in 2026). That's the difference between labor replacement through outsourcing vs. labor replacement through automation.
We are either living in different worlds, or squabbling over different meanings of words.
Models have absolutely acquired agency as of 2025. Developers are no longer copy-pasting code from ChatGPT into their text editor, they're working with agents like Claude Code and Codex that can edit code, run terminal commands, do web searches, manage their own context windows, sift through gigabytes of logs with datadog MCP, etc.
Self-improvement is also being worked on. Claude Tag learns over time in slack convos. My company also has an agent that updates its own skill files after every conversation so that we don't need to keep reminding it about the same workflows every time. Is it clunky as hell? Yes. Are the labs plowing billions of dollars into "continual learning" and "recursive self improvement"? Also yes.
What you call a model acquiring agency I call plain old software with productivity workflows designed by humans, with deliberate goals. We must separate “model” and an execution environment using a model. [Model] ≠ [A glorified shell script doing API calls in a control flow based on heuristics]. Agents are not AI, they are plain old software. The weights are the model, and that very much remains a static artifact (and pre-post training models haven’t improved much over the last few years).
What you call self improvement is a duck tape hack to imitate persistence and save on inference. Every time you do an API call, anything that needs to be processed is sent to the model. Narrowing that context down saves money. Finding clever ways to do that improves apparent performance and value. The cleverness is still human.
These are all useful innovations on top of LLMs, which remain models that generate text and symbols based on static weights, which in turn represent training data and the provider’s preferences.
You say that as if the culture difference with a truly alien intelligence is insignificant compared to the culture difference with an "overseas person".
(Even assuming "intelligent" is a sensible label to apply to an LLM holding hands with a shell script in an infinite loop)
I’m at a fully remote company with staff in at least 8 countries speaking at least 5 languages. It works out fine. A possible analogy to AI is that a lot depends on how you use it. The “skill issue” doesn’t disappear, at least not yet.
They are already producing truly astonishing results. The unit distance problem and Jacobian conjecture are world-famous problems that stood unsolved for 80 and 87 years. Entire careers have been spent on them. Yitang Zhang for example spent his entire PhD trying and failing to prove the latter in the 2-dimensional case.
> The effects of these guidelines are still being felt through the biotechnology industry and the participation of the general public in scientific discourse. Due to potential safety hazards, scientists worldwide had halted experiments using recombinant DNA technology, which entailed combining DNAs from different organisms. After the establishment of the guidelines during the conference, scientists continued with their research, which increased fundamental knowledge about biology and the public's interest in biomedical research.
Among most people that nuance will be lost. What they’ll hear is models are dangerous, so they should be controlled/regulated, by those who know best, the incumbents.
Personally, I think you're both right, bit whatever the end result is will depend entirely on the narrative that those in power chooses as the winner.
Maybe open weights models get banned, but the between-the-lines good news about that is that they'll still be available to those who know, which also means that bad banning can be overturned if and when 'those in power' are a different group.
Additionally, it might just mean that the US falls behind, bit I doubt those that are at risk of 'falling behind' would actually pay heed to a ban on the open weights models (privately at least).
Ok but the allegation is that OpenAI intentionally hacked HuggingFace as a marketing ploy. This is mental gymnastics, conspiratorial thinking that everything the incumbents say must be nefarious. And it's not clear to me that this will be the takeaway for ordinary people, as opposed to "OpenAI is reckless and can't even control their own AI."
Not as a marketing ploy. I think they were doing gain-of-function testing and intentionally had their model target HF to do a bit of pen-testing as well - HF being the site where all open models are hosted and thus OpenAI's largest nemesis after Anthropic. It wasn't like their model all of the sudden all by itself decided to do this ("Oh, noes!")- they directed it and they got caught.
Ok but Dario has been thinking about AI Safety since 2016 [1], before even GPT-1. I think the simplest explanation is that the Anthropic folks genuinely believe what they say, it just happens to also help their business a lot.
Yeah I think this is right. The best setup is when a true belief aligns with a competitive moat.
I definitely believe that (to his credit!) Amodei is a true believer in safety. But I also think it was important for many of the deep pockets investors who have been involved in the company since early on to recognize that this would be a potentially defensible moat.
What was the quote? For what it's worth, I do really think that Amodei believes in and cares about safety. But that is not the same as believing that he is entirely altruistic or above the influence of politics.
That just shows how wrong he's been because there was nothing unsafe about AI in 2016. And the people theorizing about this stuff in the 20th century? I want to see what crazy code they were writing
> Nobody is qualified to steward the development of superintelligence. It is a terrifying, unprecedented thing that our species is doing right now, and the fact that private companies aren’t the ideal institutions to take up this task does not mean the Pentagon or the White House is.
>
The only way we can preserve our free society is if we make laws and norms through our political system that it is unacceptable for the government to use AI to enforce mass surveillance and censorship and control. Just as after WW2, the world set the norm that it is unacceptable to use nuclear weapons to wage war.
Thanks for sharing, i agree no one should have absolute control of anything imo, and the mo greater the magnitude of implications the more important it should be stewarded democratically with clear and transparent principles with values adhering to things like human dignity, freedom, human, planetary & animal wellbeing etc etc. ill have to read the article
The USG has a safety organization (CAISI), but it has been neutered by the current administration (with the recent stop-work order etc.). Perhaps UK AISI would be closest to what you are looking for? See their recent work on Kimi K3 cyber (which was declared safe) [1].
It's tricky because a lot of the safety researchers have ties to the labs since those were the only companies training LLMs >5 years ago.
It is consistent with their stated beliefs, unlike OpenAI who flip flop every 6 months on whether they support open source or not.
I think their biggest PR problem is that many people still think of loss-of-control/misalignment etc. as sci-fi. And the distillation arguments come off poorly because people feel as though all the labs have trained on their creative output without their consent, so they deserve to own the result in some way.
reply