Because RDBMS can only scale as far as your most powerful machine. Unless of course you start sharding to several machines, and then you hit the same issues mentioned in this article.
I think people really overestimate their data-store needs, and underestimate how much a big iron DB server can deliver.
Its probably worth thinking about scaling out on your data end, but until you start hitting some sort of limit on a 8 core 96 gig machine with server level SSD's its probably worth investing your time in other issues.
That's a very valid point. You can get reallllly far with just one really powerful database.
But, if you've built a site on the precondition that one database is enough, with a lot of joins so you can't split tables, and your hardware can't keep up.. you're in a really bad place. I say this out of experience ;)
Just out of curiosity what sort of site would that be? I would be really interested to hear of the use cases that cause a single powerful db server to be insufficient.
Of course over a certain scale this is normal, so im not interested in a site with 50 millions users, but cases where a single DB is not the best solution.
Basically, it was a site with a medium amount of paid users. Probably 10-20k daily uniques. However, the nature and diversity of the data behind this site required thousands of tables and tens of thousands of stored procedures.
Since these queries were built over ~10 years with the expectation of being able to join any of the thousands of tables, it would now be a tremendous undertaking to perform any sort of sharding or other ways of distributing load (other than master-slave replication, which was used widely).
It's been a struggle to keep hardware up to pace with the desired growth of the company. Even with a dedicated data center and top of the line hardware, doubling their user base would probably require a significant architectural undertaking.
Had the site been designed to support an arbitrary amount of distributed database servers from day one, it would now be trivial to grow horizontally.
The scalability you get really depends on the type of queries you fire, which depends on the kind of schema you have. A good schema designed for read (assuming that read is much more frequent than a write for some use-case) rather than write would definietly work out well with a SQL db.
