One Hash Bloom Filters
44 points by pushcx
44 points by pushcx
Link was probably intended to be --
https://www.calabro.io/ohbf
I learned the Kirsch and Mitzenmacher trick from Google’s Guava. Interestingly, they also use Murmur-128, but then they just use the first 64 bits of that and split it into the two 32-bit hashes as described in the paper.
I wrote a couple of alternative implementations based on that idea, but using SipHash instead of Murmur IIRC. One as a memory-mapped fast lookup for known-compromised passwords, and another for JWT logout described in these blog posts from 2016 (written to support massive projected usage at the BBC, no idea if they still use it):
https://neilmadden.blog/2016/02/25/stateless-session-logout-in-openam-13/
https://neilmadden.blog/2016/06/06/bloom-filter-session-logout-some-numbers/
The linked paper describes a "Standard Bloom Filter" as using k hash functions and a modulo.
If I was to build a simple Bloom filter:
Perhaps those are mentioned later in the paper or in the references, but weren't in the related work or the beginning of the design section before the math got too dense for me to skim on my phone.
While there might be other e.g. locality benefits to this scheme, making a "one hash Bloom filter" seems trivial. Is there something I'm missing where the simple scheme wouldn't actually work?
Looking at Lemma 2 and Theorem 1 of the linked paper yes, you could use your sliding bit window idea to accomplish the same thing.
I really like this kind of stuff :) The combination of clever maths, cache locality, and concurrency optimisations is fascinating. I'd be curious to see how much of the improvement comes from the one-hash approach itself versus cache-line blocking and reduced contention. A comparison against conventional double-hashed and blocked Bloom filters, at the same memory budget and false-positive rate, would be interesting. Especially with varying filter sizes (inside and beyond L3 cache) and concurrent write loads.