Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You can control Google's scraping by specifying a robots.txt, which most people won't do because they want Google to index them. The article means parties who don't care about what you want.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: