The /news page ranks links based on score and time since submission. I'm only ranking links based on score (and whether it has been on a previous daily).
I could scrape the entire site at midnight each day, but I think PG would be very unhappy with me if I did that. Scaping /news every 5 minutes imposes much less load, and since highly ranked links get almost all of their votes prior to falling off the front page, this gives me almost as much information.
I'd prefer to not post the code publicly, simply because I don't want to encourage people to put extra load on the HN server -- but if you want a copy, send me an email and I'll provide it.
