It shouldn't be surprising OpenAI does have the most compute out of all the major labs. The only reason why Anthropic models are expensive is they are the most in demand models in the world and Anthropic is fighting for compute. The only way to you limit demand for your model is increasing API pricing this is also why Anthropic probably has great margin and probably is profitable compared to OpenAI.
I doubt there's any sort of criminal behavior there - the model is anthropic's up and anthropic probably charges a very expensive license fee that's the same for all of them, and their cogs on compute aren't going to be wildly different, so the main drivers of the cost are roughly the same and they're all offering customers the same end product so the prices would likely also be similar in the end
You seem terribly confused. Manufacturers are almost always able to set prices. That is not anti-competitive because it does not imply they are colluding with their competition...
No manufacturers might suggest prices, they can't set them, not in any sensible country.
If they do, they do it secretly and when governments find about that they are going to receive a big fine, together with shops that colluded with them.
The competition is between openai and anthropic, if there are price agreements between them that's absolutely price fixing. Or if there's collusion between the cloud providers to inflate compute. I would expect Amazon and gcp to both pay about the same in license fees to anthropic for their models though because they're paying for the same thing. If I buy an apple for a dollar at one store and an apple for a dollar at another store - maybe there's price fixing, or maybe that's just the cost of apples at the moment.
In capitalism there are many many competitions going on at the same time. Both models can compete and inference providers can compete for costs.
In your apple example if at the same stores you saw "open" apples having different prices and having sales you might question if the costs of those frontier apples are not being manipulated.
Yeah there's a brand premium. Like literally with apples the ones with trademarked names can cost more. And the ones with trademarked names have an organization behind them that promote that apple variety and set fees etc for growing and selling those apples. And customers are willing to pay more because those apples usually taste way better (the group exists to stop growers from enshittifying the apple by selecting for yield over flavor like what happened to honey crisp). Those prices are being "manipulated" but that's not criminal behavior - it's not illegal and wouldn't make sense to try and make illegal. The frontier models also do have different coats and have "sales" (for personal plans, the amount of usage you can get on the 200 dollar plans is orders of magnitude more than you could get for a similar cost for any open model - you'd need the same capability at the same token efficiency at literally 1/40th the cost to be able to be cheaper) - to the extent that if there is illegal stuff going on it seems more likely to me that it's on the category of dumping/pricing unreasonably low to kill competitors in some anti-competitive way (though as I understand it it's not something courts tend to find as illegal) instead of price fixing
Do you really think there is nothing someone could do to make it a fraction of a percentage cheaper to serve like having access to cheaper electricity or a more mature cloud management software. Even saving a fraction of a penny on the prices can make a different due to how much volume people are paying for.
There is a blog post waiting to be written (that I won't write) about the size/effort tradeoffs, and particularly how small models get some surprisingly good results with lots of turns and reasoning.
DeepSWE will let you chart turns taken or tokens used, and FrontierCode will chart tokens. If you use that, you can see Sol high and Terra max get about the same DeepSWE number, but Terra max takes twice the turns. Luna max scores a smidgen lower with even more turns.
Smaller models relying on lots reasoning may "scale down" better on easier tasks, because unlike size, reasoning effort is dynamic: the model can see the task looks easy and stop. On DeepSWE, the cost curves for the three 5.6 models are almost on top of each other, but on FrontierCode Extended, the version of FrontierCode with the most everyday tasks in the mix, there's a spread of costs at the ~55% level.
The recent Laguna S 2.1 model (118B, 8B active) puts up surprising coding numbers for its size, and the lab behind it specifically credits its "way of working (persistence, verification, willingness to backtrack)". Some other open models that folks report getting good mileage out of seem to get there partly by throwing a lot of reasoning at the problem.
There is a little bit of a question, if some models rely on getting it wrong a bit more at first and external checks catching the problems, of whether they're also more frequently getting things wrong they can't self-verify (say, quality of UI or API design) and then it falls to the human to find it. Still, getting the results they're getting at all is neat.
Some benchmarks historically favored reporting only on the max variants, maybe because they want to show the frontier? but that is not always what you need for practical decision. (AA has the full effort sweep for Opus 5 and Sol/Luna/Terra at least.) And at least FrontierCode finds Opus 5 taking a hit in performance above 'medium'.
I am not trying to pick a winner here. I'm probably not going to use tiny models on max for everything, but I think it's cool that you can get so much more out of a small model by amping up reasoning, tool use, and persistence.
Forgot about the ol "but how many tokens did you spend to get _there_"--wish benchmarks would include the number of input/output tokens to achieve the score. I think the closest is Android Bench https://developer.android.com/bench although best you can do is extrapolate off time/cost (iirc they claim to prefer using provider's native API)
In general, smart models work fine with any tools, dumber models need better tools to achieve same results but better tools can eat more context/take more turns
I've gotten decent results with Llama 3.1 8b on Hugging Face tester with Exa MCP since it seems to dump sufficient context into WebSearch/WebFetch type calls even a crappy model almost always gets back what it needs as long as it calls the tool at least once. I had Claude Code look at previous sessions with SearXNG vibe MCP compared to Exa MCP and results got better when it modified SearXNG to work very similarly to Exa. Ended up with this https://gist.github.com/nijave/604c43e3e0fdcd60f5280d3a6b109... although it's really only optimized for "search" not "fetch" at this point. Fetch is basic niquests without Javascript or anything clever
I hate to disagree with Broadcom Throwaway himself but it's unlikely that the OpenAI Jalapeno ASIC has been deployed yet. It takes 6-12 months to test, develop software, ramp production, etc.
