I really like this too - having the previous plan and implementation in place to create the next plan, but then clearing context once that next plan exists feels like a great way to have exactly the right context at the right time.
I often do follow ups, that would have been short message replies before, as plans, just so I can clear context once it’s ready. I’m hitting the context limit much less often now too.
I suspect it has something to do with a) the average quality of code in open source repos and b) the way the reward signal is applied in RL post-training - does the model face consequences of a brittle implementation for a task?
I wonder if these RL runs can extend over multiple sequential evaluations, where poor design in an early task hampers performance later on, as measured by amount of tokens required to add new functionality without breaking existing functionality.
Yeah I've been wondering if the increasing coding RL is going to draw models towards very short term goals relative to just learning from open source code in the wild
Maybe. Another potential, more positive, timeline is that semantically ablated content filling everyone’s feeds turns people off, and slowly kills the social feed paradigm.
That is doubtful. Most of the content on people's feed was basically converging to the same paradigm anyways. Think the Mr. Beast "I gave someone 1 million dollars to try a potato" content.
Politics is accruing and deploying political capital within an organisation - or less abstractly, building relationships and using them.
What you’re describing is a particular form of manipulative and divisive politics which is performed by insecure, desperate or selfish people.
Many engineers are not good at building relationships (the job of coding isn’t optimal for it after all), so painting the people who are good at is as narcissistic may be comforting but isn’t correct.
If LLMs stopped improving today I’m sure you would be correct- as it is I think it’s very hard to predict what the future holds and where the advancements take us.
I don’t see a particularly good reason why LLMs wouldn’t be able to do most programming tasks, with the limitation being our ability to specify the problem sufficiently well.
I feel like we’ve been hearing this for 4 years now. The improvements to programming (IME) haven’t come from improved models, they’ve come from agents, tooling, and environment integrations.
> I feel like we’ve been hearing this for 4 years now.
I feel we were hearing very similar claims 40 years ago, about how the next version of "Fourth Generation Languages" were going to enable business people and managers to write their own software without needing pesky programmers to do it for them. They'll "just" need to learn how to specify the problem sufficiently well.
(Where "just" is used in it's "I don't understand the problem well enough to know how complicated or difficult what I'm about to say next is" sense. "Just stop buying cigarettes, smoker!", "Just eat less and exercise more, fat person!", "Just get a better paying job, poor person!", "Just cheer up, depressed person!")
Both is true, models have also been significantly improved in the last year alone, let's not even talk about 4 years ago. Agents, tooling and other sugar on top is just that - enabling more efficient and creative usage, but let's not undermine how much better models today are compared to what was available in the past.
The code that's generated when given a long leash is still crap. But damned if I didn't use a JIRA mcp and a gitlab mcp, and just have the corporate AI just "do" a couple of well defined and well scoped tickets, including interacting with JIRA to get the ticket contents, update its progress, push to gitlab, and open an MR. Then, the corporate CodeRabbit does a first pass code review against the code so any glaring errors are stomped out before a human can review it. What's more scary though is that the JIRA tickets were created by a design doc that was half AI generated in the first place. The human proposed something, the AI asked clarifying questions, then broke the project down into milestones and then tickets, and then created the epic and issues on JIRA. One of my tradie friends taking an HVAC class tells me that there are a couple of programmers in his class looking to switch careers. I don't know what the future brings, but those programmers (sorry, "software developers") may have the right idea.
Yes we get it, there is a ton of "work" being done in corporate environments, in which the slop that generative AI churns out is similar to the slop that humans churn out. Congrats.
How do you judge model improvements vs tooling improvements?
If not working at one of the big players or running your own, it appears that even the APIs these days are wrapped in layers of tooling and abstracting raw model access more than ever.
> even the APIs these days are wrapped in layers of tooling and abstracting raw model access more than ever.
No, the APIs for these models haven't really changed all that much since 2023. The de facto standard for the field is still the chat completions API that was released in early 2023. It is almost entirely model improvements, not tooling improvements that are driving things forward. Tooling improvements are basically entirely dependent on model improvements (if you were to stick GPT-4, Sonnet 3.5, or any other pre-2025 model in today's tooling, things would suck horribly).
Improved tooling/agent scaffolds, whatever, are symptoms of improved model capabilities, not the cause of better capabilities. You put a 2023-era model such as GPT-4 or even e.g. a 2024-era model such as Sonnet 3.5 in today's tooling and they would crash and burn.
The scaffolding and tooling for these models have been tried ever since GPT-3 came out in 2020 in different forms and prototypes. The only reason they're taking off in 2025 is that models are finally capable enough to use them.
Yet when you compare the same model in 2 different agents you can easily see capability differences. But cross (same tier) model in the same agent is much less stark.
My personal opinion is that there was a threshold earlier this year where the models got basically competent enough to be used for serious programming work. But all the major on the ground improvements since then has gone from the agents, and not all agents are equal, while all sota models are effectively.
> Yet when you compare the same model in 2 different agents you can easily see capability differences.
Yes definitely. But this is to be expected. Heck take the same person and put them in two different environments and they'll have very different performance!
> But cross (same tier) model in the same agent is much less stark.
Unclear what you mean by this. I do agree that the big three companies (OpenAI, Anthropic, Google DeepMind) are all more or less neck and neck in SOTA models, but every new generation has been a leap. They just keep leaping over each other.
If you compare e.g. Opus 4.1 and Opus 4.5 in the same agent harness, Opus 4.5 is way better. If you compare Gemini 3 Pro and Gemini 2.5 Pro in the same agent harness, Gemini 3 is way better. I don't do much coding or benchmarking with OpenAI's family of models, but anecdotally have heard the same thing going from GPT-5 to GPT-5.2.
The on the ground improvements have been coming primarily from model improvements, not harness improvements (the latter is unlocked by the former). Again, it's not that there were breakthroughs in agent frameworks that happened; all the ideas we're seeing now have all been tried before. Models simply weren't capable enough to actually use them. It's just that more and more (pre-tried!) frameworks are starting to make sense now. Indeed, there are certain frameworks and workflows that simply did not make sense with Q2-Q3 2025 models that now make sense with Q4 2025 models.
I actually have spent a lot of time doing comparisons between the 4.1 and 4.5 Claude models (and lately the 5.1->5.2 chatgpt models) and for many many tasks there is not significant improvement.
All things being equal I agree that the models are improving, but for many of the tasks I’m testing what has the most improvement is the agent. The agents choosing the appropriate model for the task for instance has been huge.
I do believe there is beneficial symbiosis but for my results the agent's provide much bigger variance than the model.
LLM capability improvement is hitting a plateau with recent advancements mostly relying on accessing context locally (RAG), or remotely (MCP), with a lot of extra tokens (read: drinking water and energy), being spent prompting models for "reasoning". Foundation-wise, observed improvements are incremental, not exponential.
> able to do most programming tasks, with the limitation being our ability to specify the problem sufficiently well
We've spent 80 years trying to figure that out. I'm not sure why anyone would think we're going to crack this one anytime in the next few years.
> Foundation-wise, observed improvements are incremental, not exponential.
Incremental gains are fine. I suspect capability of models scales roughly as the logarithm of their training effort.
> (read: drinking water and energy)
Water is not much of a concern in most of the world. And you can cool without using water, if you need to. (And it doesn't have to be drinking water anyway.)
Yes, energy is a limiting factor. But the big sink is in training. And we are still getting more energy efficient. At least to reach any given capability level; of course in total we will be spending more and more energy to reach ever higher levels.
Incremental gains in output seem to - so far - require exponential gains in input. This is not fine.
Water is a concern in huge parts of the World, as is energy consumption.
And if the big sink is “just” in training, why is there so much money being thrown at inference capacity?
I thought it was mad when I read that Bitcoin uses more energy than the country of Austria, but knowing AI inference using more energy than all the homes in the USA is so, so, so much worse given the quality of the outputs are so mediocre.
It seems to me that MCP and Skills are solving 2 different problems and provide solutions that compliment each other quite nicely.
MCP is about integration of external systems and services. Skills are about context management - providing context on demand.
As Simon mentions, one issue with MCP is token use. Skills seem like a straightforward way to manage that problem: just put the MCP tools list inside a skill where they use no tokens until required.
Just consider what it fundamentally is: a company at the leading edge of a product category that has found absurdly strong technology/use-case fit, and is growing insanely fast.
Looking for a moat in the technology is always a bit of a trap - it’s in the traction, the brand awareness, the user data etc.
> Looking for a moat in the technology is always a bit of a trap - it’s in the traction, the brand awareness, the user data etc.
Traction, brand awareness, and user data do not favor Windsurf over GitHub Copilot. The few of us who follow all the new developments are aware that Windsurf has been roughly leading the pack in terms of capabilities, but do not underestimate the power of being bundled into both VS Code and GitHub by default. Everyone else is an upstart by comparison and needs some form of edge to make up for it, and without a moat it will be very hard for them to maintain their edge long enough to beat GitHub's dominance.
Definitely take that point. But this valuation is perhaps more about how much that traction, brand and data is worth to OpenAI, who cannot buy Copilot. $3bn doesn’t seem so disproportionate in that context especially given the amount of money being attracted to the space.
Define losing? My company pays for Copilot but not for Cursor, and it's not at all clear to me that we're the exception rather than the norm. What numbers and data are you working with?
That's not actually how unseating an incumbent works. The incumbent can adapt to the threat for quite a while if they act on it, they just have to not be Blockbuster. Copilot is showing every sign of making up ground feature-wise, which is bad news for the runners up.
Incumbent advantage of being in VS Code already? Thing is, Cursor is basically just VS Code, there's hardly any barrier to switching, so it's quite a weak advantage.
In brand velocity maybe, but copilot is rapidly reaching feature parity with cursor and will invariably overtake it—while costing less to users.
Same with Google vs OpenAI. I tend to agree with the sentiment that I most frequently hear which is that OpenAI is the currently popular brand, but that can only carry them so far against what will eventually be a better offering for cheaper.
Yeah it's very interesting... It appears to lead itself astray: the way it looks at several situational characteristics, gives each a "throw-away" example, only to then mushing all those examples together to make a joke seems to be it's downfall in this particular case.
Also I can't help but think that if it had written out a few example jokes about animals rather than simply "thinking" about jokes, it might have come up with something better
reply