I suspect that if a crawler is being used to farm email addresses for spamming, it's highly unlikely that robots.txt would be any deterrent whatsoever.
Spammers' crawlers use URLs obtained from search engines and public sources. If the whole directory is blocked in robots.txt, it WILL reduce crawling activity massively.
