>The key addition with Pilosa is that it's distributed and can scale horizontally :)
Doesn't this also make it several orders of magnitude slower? Every time I find myself using Bitmaps it's when speed is extremely important. What are some use cases for this not covered by in-process Bitmaps, Bloom filters, and HyperLogLog?
I'm just not aware of many use cases for bitmap indexes willing to trade that much speed for database-like access. I think this project would be more usable factored into a library than something external
The use case for Pilosa is usually as an addition to something like Cassandra or HDFS where you have terabytes (or more) of data that you want to be able to query more flexibly - especially if there are a very large number of attributes that you want to filter and segment on (think tens of millions).
I think you're right though, that there are use cases which would benefit from a library exposing this functionality - you need to have quite a lot of data before compressed bitmaps representing the relationships in that data start overflowing memory on a single machine.
The key addition with Pilosa is that it's distributed and can scale horizontally :)