Over and over again they talk about how Anthropic's margins are getting squeezed. They're so squeezed in fact that Anthropic has merely 85% gross margins! They're earning merely $6 in revenue for every $1 they spend!
It's 85% gross margin only if you divide by "Compute and infrastructure", which does not include personnel, legal settlements, sales, marketing and partnerships, or other operating and administrative. Their total operating expenses (34Bn as of 30th September this year) is slightly higher than their revenue (33Bn as of 30th September this year) which means they have an operating loss of ~0.6Bn so far this year. The majority of the negative numbers lie in debt to Amazon and Google (72Bn + 8Bn other), which is actually mostly convertible notes that to my understanding are never going to be paid back as cash, but as stocks in the company. A valuation of 1-2 trillion would give them plenty of room to pay back that debt with stocks, so imo. that part is a little overblown. IPOing would directly help reduce these numbers.
The spending commitments the article mentions are a little more concerning, but I haven't looked into that so I'm currently taking that with a huge grain of salt (especially since the article felt very biased and politically motivated).
CPU DRAM can't really be used for inference efficiently -- inference mostly wants memory bandwidth, not memory capacity, and GPU DRAM has >10x more bandwidth. The fabs can switch between them but you can't switch after the fact.
Those machines with GPUs still need RAM of their own, and they generally want large caches to avoid SSD penalties. You even see this spill out in the form of costs for KV cache in <1min, 5m, 1hr rates etc.
This is just wrong unless you're using some confusing definition. Notice that companies trying to do lots of inference aren't looking for CPUs, they're looking for GPUs/TPUs.
Ah well, the opposite actually. Your definition, while traditional, is wrong. It was right some time ago. But it's wrong now. At least according to Intel and Nvidia in their disclosures. Intel was literally talking about their sales to "companies that do lots of inference" when they said it... so...
They have various benchmarks, e.g. how much time it takes them to do wikipedia page -> page games. Jev seems to take the same or fewer hops but in ~10x less time and for ~10x less money.
It's totally reasonable to compare against LLMs doing chain of thought if it gets comparable performance.
To what degree can you say something like "if the kernel doesn't have metaprogramming [or some other set of features] it's fine". When were the last bugs with a reduced feature set?
Read the article. The elaborator (where metaprogramming is evaluated) is not part of the trusted computing base. The security model is that bad proof terms get rejected, not that they're never generated.
I like ast-grep and I think the changes are interesting, but it isn't fun reading an ai-generated article.
It seems like a lot of the speedup comes from using an arena, using realloc with offsets rather than raw pointers and some change to the graph that enabled this (though I don't want to re-read it to confirm).
The correct comparison is not sonnet, but qwen3.5-27b on a cloud. Alibaba's pricing [0] is $0.20/m input $1.56/m output, so $0.72 for the prompt processing or $0.22 for the generation. Yours is still cheaper but the margins are less.
My guess is that this math gets less good with MoE (because you will be limited by VRAM, but clouds won't).
reply