Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It doesn't seem to be mentioned anywhere in the paper, but this is a description of the "consistent hashing" algorithm in Google's Guava library: http://docs.guava-libraries.googlecode.com/git/javadoc/src-h...

I find it kind of funny that they created and released an apparently novel hashing method, and then waited 2.5 years to actually explain how it works.



Hm I think you are right. The Guava code is apparently the simpler version of the algorithm that takes linear time (but still no memory). (Apologies, it appeared to be different at first and I downvoted you; wish I could undo it).

It seems likely that the new part is the optimizations. Still, it is pretty interesting as I didn't think Guava overlapped with this area and these authors much within Google (I work there). It's usually inaccurate to speak of Google as one entity with specific reasons for doing things a certain way; there are many people working on many different and overlapping things :)


It takes time and effort to do a writeup. And it is easy to procrastinate. That reminds me...

A bit of a plug here. I have a cute nearly-minimal perfect hashing algorithm designed to have good cache-friendly properties. Briefly, it is somewhat similar to hopscotch hashing, only you pre-calculate the positions of the elements to put them into the 'best' spots by solving the assignment problem. Works for up to about 50k elements. It feels like it might have good theoretical properties too, might be even optimal, but it was a while since I've taken the algorithms class.

If anyone is interested to do a writeup and publish clean source code - you'd be welcome.


It sounds similar to robin hood hashing. Is there source code anywhere?


Yes, it is similar to robin hood. Only you actually place items into optimal positions (by solving the assignment problem on your memory/cache access costs & access probabilities), rather than stochastically swapping items.

I'll put up sample code if somebody would be willing to do a writeup ;)


I first read that as prohashtinate.




Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: