If you don't mind me asking a question to better understand your perspective: Are you old enough to have personally experienced and built self-hosted things before the advent of AWS, cloud hosted stuff, or is your text above (which I do not disagree with in any way) the point of view of a person who began working on this stuff in the "cloud-native" era, post-2012 or around then?
I'm thinking from the "20 years ago" it is the first one, so your experience is even more valuable for those younger persons reading it because you've seen how things were before the vendor-lock-in era began.
I ask because I've been seeing an increasing trend of people who've self-taught themselves Linux/BSD things, grew up in the cloud-native era, and have now become increasing dissatisfied with the "Do everything in one vendor cloud" operations model that they initially learned. Many are now going the opposite direction and deciding to self host their own projects at much lower cost and much less vendor lock-in.
I'm in my 40s, full disclosure I've never worked in devops 'officially' but have hosted many personal and commercial projects, worked as a developer and later product manager. I have run my own small businesses as well as working for various corps.
In defence of cloud I will say that back in mid 2000s I did have to buy and rack my own servers, this was a pain in the ass and I don't want to ever see and especially hear the inside of a datacenter again. I'll also say that a lot of things are much easier and safer today, the industry has developed a lot of process and tooling.
So the issue isn't necessarily "cloud" but how large cloud vendors have strategised in their own best interests to do vendor lock-in, especially on the lower end of startups that are obviously open to shipping now and paying later, and how we have all lapped that up because they do give generous free-tiers and they do solve some real problems.
Yeah, self-hosting is definitely having a come back. Lots of content creators who would have been evangelizing cloud solutions in the past have been instead going into the various self hosting rabbit holes. There is a bit of a pendulum swing. You now have access to so much information and tooling to host your own things, it’s pretty awesome compared to what we had before 2010.
Not sure how much impact that will have for companies, that will likely stay more niche. Companies will still want to delegate much of the effort to a cloud provider, but with pressure to have less lock-in
I've worked for a corp that spent good money to mirror their entire Salesforce setup internally on the basis that Salesforce was expensive, slow, and had all their customer data and could bring the entire business to its knees.
I've worked for another corp that spent insane amounts for a CMS that nobody understood and required a full time engineer on duty to "unlock" files for the content team - this company later built a simple CMS that just stored a title, body and image in a database because I shit you not: thats 90% of what they needed from a CMS and to do quick announcements/news in a mobile app would have taken weeks longer to develop if they had to integrate with the real CMS.
This was a few years ago before LLMs btw, so I can only imagine how crazy it is today
Kind of unavoidable now for hosting companies purchasing new hardware, much as diesel costs are affecting trucking companies.
Let's say theoretically that a company decides to buy raw rack space+power colo and DIY your own router setup, upstream IP transit connections and buy their own server hardware... You're suffering equally from the increase in RAM costs buying your server hardware much the same as if you go with a dedicated bare metal hardware hosting company.
Every project I've ever worked on - sans two at IBM post Red Hat acquisition - had legacy style Javaboxes at the customer's DC or was still rocking mainframe DB2 stuff. I'm seeing bare-metal Kubernetes / OpenShift gaining a stronger foothold though. Rarely, some specific workloads did run on Azure - never anything critical though.
For context, I do mainly development work for EU enterprises and the public sector.
There's always been vendor lock-in. Someone has to implement technologies to the standards, those are most of the time commercial interests.
Apache vs nginx, rhel vs debian, spring vs j2ee, etc. etc.
I think it's good for the young crowd to get into self-hosting. You learn more fundamentals that way, for sure. But I'm not looking to move stuff into self-hosted.
I get unlimited scalability in AWS. I can standup and run an app for $0-$5/month depending on how I want to build it. That app will have unlimited scalability, redundancy, automated backups, etc. It's not worth dealing with the headaches of managing things myself.
Most everything is the way it is today for good reason, it's just that the youngsters haven't had to learn those reasons themselves.
I remember driving to the datacenter to setup my own servers. I had a fractional T1 to my desk, it provided a blazing 640k (sometimes all the way up to an unfathomable 1MB!!1!) of bandwidth and only cost around $1,300 a month. Good times.
That gave way to hosting providers, like ServerBeach and Rackspace, where you paid them to setup the server hardware for you, saving the drive. You still had to deal with bandwidth, backup, redundancy, etc. Much easier than before but still a pretty big hassle.
Then the cloud happened. "Come to the cloud, where everything just magically happens for you," the salesperson said. So I went. And discovered that none of my scripts, tools or knowledge worked there, and had to completely start over from scratch. FOR EACH VENDOR. Gah.
The ONE thing I will grant cloud providers is that I can usually automate my way around their enshittification. They over-complicate things to an extreme where I do wonder if there is an actual savings or not, just based on the headaches their setups can consistently produce.
My take is that your needs pretty much determine which route is more/less painful. Sometimes you DO need the redundancy and uptime enough to tolerate the shenanigans of the vendor, and other times you just need your own machine to quietly get things done (without worrying about SLAs or client expectations).
The ultimate question you should be asking, and there is no right/wrong answer: For this particular effort, is it more valuable to you to learn how to do these things by hand, or is the infrastructure just a means to an end so you can do something specific?
Yeah agreed, there are pros and cons but ultimately we are exchanging control for convenience.
The idea of a hosting provider that manages all the boilerplate, security updates etc is not a bad thing. The idea of being able to spin up another node, have load balancing etc just work and you didn't even have to get out of bed - is also not a bad thing.
Where it becomes suspicious is when Amazon and Google both have their 'own' databases, which are just different enough to make it a pain. And yes I know I can spin up e.g. a Postgres instance on Google or AWS for not much money, but now I have a vendor-specific provisioning script. "Just use Terraform or Pulumi" - even when an LLM manages this web of corporate enterprise nonsense it's still unmanageable and everyone just ends up with a run book of prayers.
There’s a big gap between hobby self-hosting and enterprise.
Clouds are appealing (note, not technically better but more appealing) in the enterprise for numerous reasons. We already have a contract, I don’t have to spend a month with procurement. Their support is “respected” so “I asked AWS and they told me to pound sand” is an acceptable response to a lot of requests. No one ever got fired for picking AWS. The list goes on.
None of those friction points apply to hobby hosting, really. IAM is annoying because the alternative isn’t “fill out 7 forms and host 12 meetings to get a new vendor in”.
You also get less of the benefits. Low volume SQS can be trivially replaced, but if you use enough that you’re debating making a whole team to manage Kafka then paying the cloud tax can look appealing because of the predictability (your Kafka team might fail, SQS probably handles larger workloads currently).
At some scale, not having to wait 12 months for the message queuing team to support TLS for some regulation becomes worth a cloud tax.
I'm thinking from the "20 years ago" it is the first one, so your experience is even more valuable for those younger persons reading it because you've seen how things were before the vendor-lock-in era began.
I ask because I've been seeing an increasing trend of people who've self-taught themselves Linux/BSD things, grew up in the cloud-native era, and have now become increasing dissatisfied with the "Do everything in one vendor cloud" operations model that they initially learned. Many are now going the opposite direction and deciding to self host their own projects at much lower cost and much less vendor lock-in.