If their new platform reduces inference token cost by 10x, does that play well or not well with the recently updated GPU deprecation schedules companies have been playing with to reduce projected cost outlays?
For context, my understanding is that companies have recently moved to mark their expected GPU deprecation cycles from 3 years to as high as 6 which has huge impacts on projected expenditures.
I wonder what the step was for the Blackwell platform from the previous. Is this slower which might indicate that the slower deprecation cycle is warranted, or faster?
No way you throw away Blackwell GPUs after just 3 years. Google runs 8 year old TPUs still at 100% utilization. Why would you depreciate them in just 3 years?
The conversation around GPU lifecycles seems to be conflating the various shear rates within the data center. My layman understanding is that the old 3 year replacement cycle had more to do with some component, not necessarily the memory or the processor, going wrong for half of their units by 3 years, at which point GPUs were cheap enough and advancing faster enough that it was more cost effective to upgrade than to fix. However, that calculus changes completely when the GPU and the HBM are orders of magnitude more expensive than the rest of the system. I suspect that we will see repairs being done on on the various brittle bits of the system and the actual core expensive components will continue to operate much longer than 3 years.
For context, my understanding is that companies have recently moved to mark their expected GPU deprecation cycles from 3 years to as high as 6 which has huge impacts on projected expenditures.
I wonder what the step was for the Blackwell platform from the previous. Is this slower which might indicate that the slower deprecation cycle is warranted, or faster?