I only hear bad things about bots lately, are there any benevolent bots?
I've done a tiny bit of scraping myself (usually to archive old websites before they die), and I've gotten banned a few times even though I wasn't hammering anything.
Looks like it's getting harder and harder to do scraping for legitimate purposes. Although I suppose legitimate is subjective :) Everything becomes valuable when it's old enough...
I only hear bad things about bots lately, are there any benevolent bots?
I've done a tiny bit of scraping myself (usually to archive old websites before they die), and I've gotten banned a few times even though I wasn't hammering anything.
Looks like it's getting harder and harder to do scraping for legitimate purposes. Although I suppose legitimate is subjective :) Everything becomes valuable when it's old enough...