Time is coming when you'll be able to select out of hundreds of models and they'll be downloaded on demand if not already, would totally infer locally and offline even on your phone and I guess no more than 2030, we'll be there.
I am not so up to date with the hardware landscape but don't think smart people would let not be noticing the need.
Cloud models will always have edge compared to local models. Maybe in 2030 your iPhone would run GPT-4 on your phone but cloud GPT-9 will solve all your kids homework, do 95% of your job and manage your household.
Transformers still didn't hit their scaling limits wrt number of tokens trained on and the number of layers. Model size limits are now purely given by finances ($100M/training run still seems to be too excessive).
I am not so up to date with the hardware landscape but don't think smart people would let not be noticing the need.