Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I would like to see something that helps create useful, specific scrapers for languages like Ruby and Python.

It's annoying to have to run scripts multiple times, tweaking it after each run to get exactly what you need. It's a waste of time...



  >> ipython

  [In 1]: from pyquery import PyQuery as pq
  [In 2]: pq("http://www.foo.com")("<some jquery selectors>")
(inspect output, repeat till right)

... or do it with requests + lxml.etree, or whatever you want

when you have what you need, copy and paste into a file


PERFECT! Thanks a lot =-)




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: