I interviewed somewhere with an ex-Googler who revealed they cache all of the search results for terms they have seen before, then when they update the index they also update the cached results. From that perspective, fast search results aren't actually that exciting since you can constantly run a background task to update the cached results and just serve those as the requests come in. This caching and response time seem orthogonal to the speed of the actual act of calculating search results.
I work on Google Search infrastructure. Of course, caching is important, but a large percentage of queries don't hit the cache. The backend that serves results that aren't cached has very strict latency bounds.
> From that perspective, fast search results aren't actually that exciting since you can constantly run a background task to update the cached results and just serve those as the requests come in.
If that's how it worked, I agree, it wouldn't be that impressive (every search result would just be a 1-1 cache lookup). That's not how it works, though, and as someone who works adjacent to the system, it is pretty impressive how fast it is when the work it's doing is actually pretty expensive.
For sure, but the approach is quite viable. If 19 out of 20 searches by a user are almost instantaneous and single novel one requires a few seconds, they'll assume a hiccup in their internet connection and still view the site as "really fast". It's certainly useful for limiting demands on expensive hardware.
The other 20% also perform well. I have quite literally never encountered a Google search that took more than some tens of milliseconds for a round trip.
You're absolutely correct. Caching common search queries allows the site to allocate hardware for processing "expensive" queries, with the objective of having both complete at near the same time.
Without caching, the cost of operating the site would dramatically escalate.
I recall reading about the local transit department which had gotten some fancy accelerator card to help with route searches, ie when a customer wanted to go from A to B, which busses, trams etc to take.
Can't recall if it was "just" a bunch of FPGAs but it was a big-ass PCI card.
Some years later I tried to find this story again, and to check if they still used it. Turned out they had ditched it after just a couple of years. As memory sizes had increased, they could just precompute all possible routes for the next day and keep them all in memory...
