They aren't crawling, just noticing what pages clients who visit google.com?q=xxx go to next.

If anybody's search toolbar checks a site's robots.txt before sending clickstream data, I would be very surprised.

A client-side robots.txt rule would also make anti-phishing features trivial to bypass...just put a robots.txt on your phishing site.