Why? GPUs are replaced every 3 to 5 years. This is going to be an ongoing operational cost forever. It will probably increase more if larger models require bigger VRAM sizes.
We have probably hit a limit to scaling LLMs through raw parameter count alone, at least we're not seeing the exponential pace. I personally think we'll end up with a nice sigmoid curve plateauing in the sub 10T parameter regime. The amount of tokens processed (in inference) is increasing exponentially though (I've been following open router usage stats for years and it's always been exponential). We will of course make technological advances in hardware efficiency, and model parameter efficiency, but I think a much more plausible future is that VRAM needed for loading and serving individual models will slow down or even stop. We will need more chips, and more power, as demand continues to grow of course, but the operational lifetime of GPUs today will be a lot longer than the SoTA cards from 5 years ago.
They are building new datacenters for the AI demand, so around half of this CAPEX is not for the GPU-s, and those will not be replaced every 3-5 years.
Also, TPUv2 was introduced in 2018, and still not completely retired in all regions, from accounting pov, it has been written down to 0, but they are still working.
The upfront cost of facility is ~30%, network infra 10-15%, land/utilities is small percentage, power could be significant for an AI DC. The servers are ~50-60% only.
That cost has always been there and allowed for their lucrative margins. It's the upfront cost of building/populating their datacenters (many more than before) that is eating those margins.
I mean the issue is scaling, the worlds for cloud never kept getting bigger and bigger and compute scaling had stopped a while ago in the CPU space.
With AI every new generation with both massive hardware and software stack changes from Nvidia makes prior chips extremely inefficient to run, basically we are comparing an ASIC industry to a general purpose compute industry where all work loads are the same shape and size and so on.
Margins for ASIC based mining companies or ASIC solutions providers were never high, Optane and other weird solutions are niche and great for a specific category or moment in time, but they become obsolete pretty quickly.
The fear is we don't know if this Capex can stop.
The worst type of fear is if this Capex will stop then what? Someone is very overpriced in this market, the cloud companies, the hardware providers or both.
I don't see how we reconcile this without a massive wave of repricing, ofc markets can stay irrational and we don't see the actual books but AI doesn't have so much revenue. Suddenly the AI token/cloud revenue won't 100x in a year or two...
Especially when intelligence will continue to get cheaper, the margin compression is a massive risk.
All the data centers for hyper scalers were a miniscule part of their story the real moat was the software layer on top otherwise Hetzner would be priced like an Amazon as well.
Something is shaky with this market I don't know what it's very opaque even as an insider working on for big tech and startups. I have no clue who falls first and which bottleneck cracks but there is not enough revenue for tokens, we will see a strong 2-3x growth in the next few years, from here which is absurd, but it's not enough, not nearly enough. If the capex keeps high and increasing.
Ofc they can stop the capex and the otherside gets repriced it's not like nvidia, micron and co aren't worth trillions.
I don't care about sports betting. I care about government blocking websites just because it exists on the internet. It was already illegal for Polymarket to accept bets from French people.
FanDuel cannot operate in France but their website isn't blocked. So how do we explain this?
France, EU, Italy, Spain, La Liga are just consistently making fools out of themselves trying to block everything on the internet when they don't like it.
Maybe it's just because regulatory censorship is inconsistent and doesn't make any sense. It's like the China firewall but even dumber and less effective.
So if something is illegal you don't think it's moral to enforce that people don't do it because just being illegal should be enough?
Note there's no liability on the ISPs, they do not have to pay a fine for not having blocked it, they do not have to ban alternative DNS. In Europe there is apparently a legal mindset that you can recruit a third party to conduct an action that is not onerous to enforce a blockade. A court can point to someone who has a DNS server and order them to block certain lookups.
Your entire argument is based on the fact that France cannot fight all websites at once? Will you change your opinion if Fanduel and others are blocked in 6 months as well?
A fair amount of ML/AI innovations came out of the market in general. Neural networks are a useful tool to solve a variety of problems... LLMs specifically were a more recent interesting market to develop but I've yet to see anything that could give a market player a real competitive advantage. It feels like we just invented a new hammer and now that we know how to build it it isn't that hard to build one yourself. The all purpose hammers are, of course, unreasonable to build - but those don't seem to be that useful. I don't really need Claude to be able to generate sonnets when I'm programming so I think specialization is the place we'll see genuine markets form.
The company was VC-funded as a search engine but by the time they made significant investments in AI (DeepMind etc) they'd been a publicly held company earning multiple billions a year from advertising for a decade.
Google bought DeepMind and their other major AI acquisitions. Public companies make corporate venture investments for very different reasons than LP-backed VCs. They do early-stage investments to search for emerging players they can buy as soon as possible or to gain market intelligence on trends. They do later stage investments to help grow future vendors or customers and sometimes to foster ecosystems that form their competitive moat.
But if they think it's important to their core business, corporations don't want to invest, they want to buy. Source: I used to be involved in corporate venture investing at a top 10 valley tech leader.
If your definition of VC is "Literally anything that requires a long term investment of money" then sure, but I think most people mean something different than that.
Hard to imagine that a 24 rack of coca cola is actually cheaper than an iodine tablet or a couple drops of household bleach or you know, boiling water.
It can be when you don’t have access to boiling water, bleach, iodine tablets, and/or the water you have access to is extremely contaminated with chemicals because of pollution.
Building an application that is immune to attack defeats all possible attack scenarios, yes. But no application is perfect or immune to all possible attacks, the concept of defense-in-depth is an acknowledgement of this.
Right, but what is presented here is not defense in depth. It's just an additional layer that doesn't address the issue.
The problem at hand is that the PostgreSQL session user has too many access rights and thus the SQL injection is able to wreck havoc.
The presented solution doesn't help because the SQL injection could probably bypass it easily through some other SQL way (custom triggers or rules to intercept the backend pepper or final tokens for example).
The right way to go about this is to create a schema or table per entity combined with authenticating a custom SQL user per customer and setting the RBAC rights for that user correctly.
I don't know how you can say that when some of their most recent well received games include Tomodachi life and Rhythm Heaven. Those aren't the kinds of games made from those maximizing bean counters.
Yes. And an environment like Microsoft wouldn't think a dormant, niche franchise like that would be able to achieve an overwhelming success after 2 generation.
But sure, if you want the last new IP, Nintendo has played it safe with Arms and focused more on bolstering dormant IP's on the switch. With Animal Crossing and Zelda in particular finding new winds. Splatoon was their last huge success as a new IP in 2015.
Labo came in 2018, but I have no idea how to evaluate the success of that.
I find it really sad that the GameCube and N64 were where they were the most innovative and also happens to be where they floundered the most in sales.
Huh? They are trying a lot of things and when they want to establish a new franchise they are often successful, see e.g. Splatoon. Innovation also does not correlate with new IPs.
"Trillion dollar company will definitely make tens of trillions of dollars in AI revenue but no, sorry, it can't pay a few thousand dollars to authors of content they trained on."
reply