Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
redorb
on Nov 11, 2008
|
parent
|
context
|
favorite
| on:
Ask HN: How Much Can I Scrape?
1 request per a minute I don't think yahoo would even notice.
maximilian
on Nov 11, 2008
|
next
[–]
1 request per second would be still be pretty slow. Computers think in nanoseconds. The internet thinks in milliseconds.
aneesh
on Nov 11, 2008
|
prev
[–]
True, but they might notice 1440 requests in the same day from the same IP.
flashgordon
on Nov 11, 2008
|
parent
[–]
actually the way i worked around this with both yahoo and google was to roundrobin my requests over a list of 1500 publicly available proxy servers :D
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: