Yes. There's a lot of yak, and every crawling task reveals new ones.
As an example, all of blogger is behind a single load balancer, with a rate limit. If you don't crawl blogs, you'd never know. Or the top million, plenty of blogger blogs in the top million.
As an example, all of blogger is behind a single load balancer, with a rate limit. If you don't crawl blogs, you'd never know. Or the top million, plenty of blogger blogs in the top million.
Ditto for Shopify.