Sizing Bloom Filters on Valkey: The 128 MB Cap, Scaling Math, and VALIDATESCALETO

A Bloom filter on Valkey looks free until the day BF.ADD returns ERR operation exceeds bloom object memory limit. In valkey-bloom 1.0.1, the module behind the BF.* commands, every filter is one key with a per-key memory cap of 128 MB. The first BF.ADD on a new key quietly creates a filter sized for 100 items, which then doubles each time it fills. Each doubling adds memory, adds hash checks to every lookup, and loosens the overall false-positive rate. A default filter stops growing at 26,214,300 items. A filter you sized yourself can hold several times that in less memory. This guide gives the sizing math, how scaling filters grow and stop, the NONSCALING and VALIDATESCALETO options, replication and persistence, and two patterns.

Scope and review date: November 2026. This post covers valkey-bloom 1.0.1 on Valkey 8.1 and later (8.0 needs the module built with its valkey_8_0 feature flag), the version that valkey-bundle 9.1.4 (Valkey 9.1.3) ships. Defaults, error strings and commands were checked against the valkey.io documentation and the module's 1.0.1 source on 10 November 2026. The sizing figures are arithmetic from the module's own formulas, not measurements: the commands were not run in a lab, so confirm each number with BF.INFO on your own server.

How many bits and hash functions does a Bloom filter need?

A Bloom filter is a bit array plus k hash functions. Adding an item sets k bits, and a lookup reports "maybe present" only if all k bits are set. A filter can give a false positive, but never a false negative. The valkey.io Bloom filter topic describes the behaviour. The sizing comes from the Rust bloomfilter crate that valkey-bloom 1.0.1 builds on, which uses the textbook formulas:

• bits per item = −ln(p) ÷ (ln 2)², where p is the false-positive rate;
• bitmap bytes = ⌈n × −ln(p) ÷ (8 × (ln 2)²)⌉ for n items;
• hash functions k = round(bits per item × ln 2), with a minimum of 1.

# Worked arithmetic: bytes and hash count for n items at rate p
python3 -c '
import math
n, p = 50_000_000, 0.001
bits = -math.log(p) / math.log(2) ** 2
print(f"bits/item={bits:.3f} k={round(bits * math.log(2))}")
print(f"bitmap bytes={math.ceil(n * bits / 8):,}")'
# bits/item=14.378 k=10
# bitmap bytes=89,859,923

Interpretation: 50 million items at 0.1% need about 85.7 MiB of bitmap and 10 hash functions. valkey-bloom adds a few hundred bytes of fixed overhead per filter: 264 bytes for a one-filter object, by the 1.0.1 struct sizes. That is why the documented default filter (100 items at 1%) reports a Size of 384 bytes: a 120-byte bitmap plus 264.

Each tenfold cut in the false-positive rate costs another 4.8 bits per item. The RedisBloom BF.RESERVE page gives the same bits per item but rounds the hash count up, so it lists 14 hashes at 0.01% where valkey-bloom's crate rounds to 13.

What does BF.ADD create when the key does not exist?

BF.ADD, BF.MADD and BF.INSERT create a filter on first use from the module's defaults. The topic page lists them: capacity 100 (bf.bloom-capacity), false-positive rate 0.01 (bf.bloom-fp-rate), expansion 2 (bf.bloom-expansion), tightening ratio 0.5 (bf.bloom-tightening-ratio) and a random seed (bf.bloom-use-random-seed). Expansion 2 means every implicit filter is a scaling filter.

127.0.0.1:6379> BF.ADD seen:emails user@example.com
(integer) 1
127.0.0.1:6379> BF.INFO seen:emails CAPACITY
(integer) 100
127.0.0.1:6379> BF.INFO seen:emails MAXSCALEDCAPACITY
(integer) 26214300

Interpretation: the filter starts with room for 100 items and can grow to 26,214,300 before the next sub-filter would break the 128 MB cap. Changing the defaults with CONFIG SET affects only filters created afterwards. Existing keys keep the capacity, rate and expansion they were created with.

Creating the filter explicitly with BF.RESERVE key error_rate capacity avoids the default. Add NONSCALING or EXPANSION n, but not both: valkey-bloom rejects the pair, while RedisBloom's syntax lists both options.

How does a scaling Bloom filter grow, and what does each step cost?

When the newest sub-filter is full, the next new item triggers a scale-out. valkey-bloom appends a sub-filter with capacity = previous capacity × expansion and false-positive rate = p × tighteningi, where i counts the sub-filters that already exist. With the defaults, the sub-filters hold 100, 200, 400 and so on items, at rates of 0.01, 0.005, 0.0025 and so on. Earlier sub-filters stay in the key and are checked on every lookup.

Each sub-filter adds three costs:

• Memory per item rises. Each halving of the rate adds 1.44 bits per item, so sub-filter 18 uses about 34 bits per item against 9.6 in the first.
• Lookups get slower. A lookup checks the sub-filters in turn and stops at the first one that matches. An item that is not in the filter, the common case for a guard, checks all of them. The topic page puts add and exists at O(N × K) for this reason.
• The real false-positive rate drifts up. BF.INFO still reports the rate you set, 0.01. A miss can match any sub-filter, though, so once they fill, the combined rate approaches the sum 0.01 + 0.005 + 0.0025 + … , which stays just under 0.02. With tightening 0.5, the bound is twice the requested rate.

A scale-out can also be refused before the cap, if the next sub-filter's rate would round down to zero. valkey-bloom then returns ERR false positive degrades to 0 on scale out. With the default expansion, the memory cap comes first. In practice, you meet this error only with EXPANSION 1 or a very low tightening ratio.

Where is the 128 MB per-filter cap, and can you raise it?

The cap is the module configuration bf.bloom-memory-usage-limit, default 134,217,728 bytes, and it applies to each filter separately. valkey-bloom checks it when a filter is created and before each scale-out. An operation that would exceed it fails with ERR operation exceeds bloom object memory limit and changes nothing. It is not a server-wide budget: maxmemory still governs the instance, and the writing BF.* commands carry the deny-oom flag.

127.0.0.1:6379> CONFIG GET bf.bloom-memory-usage-limit
1) "bf.bloom-memory-usage-limit"
2) "134217728"
127.0.0.1:6379> CONFIG SET bf.bloom-memory-usage-limit 268435456
OK

Interpretation: raising the limit to 256 MB takes effect immediately. The 1.0.1 code reads the current value on every create and scale-out, so a filter that was refused can grow again. A default filter under a 256 MB limit can reach 52,428,700 items in 19 sub-filters. The topic page says the limit exists to protect serialization performance.

To make it permanent, put it in valkey.conf after the loadmodule line. Keep comments on their own lines.

loadmodule /usr/lib/valkey/libvalkey_bloom.so
# Per-filter memory limit: 256 MB
bf.bloom-memory-usage-limit 268435456

Interpretation: Valkey 9.1's config parser treats only lines that start with # as comments. For a module config, it joins every token after the name into the value. The valkey-bundle topic page's sample line bf.bloom-memory-usage-limit 134217728 # 128MB therefore passes 134217728 # 128MB as the value, which is not a number. The module path is valkey-bundle's.

A non-scaling filter is one bitmap, so its maximum size under a given limit is plain division. These figures are worked arithmetic from the 1.0.1 formulas.

Should you reserve a scaling or a NONSCALING filter?

The topic page's rule: use a non-scaling filter when the item count is known and fixed, and a scaling one when it is not. The panels load the same 26,214,300 items three ways, as worked arithmetic from the 1.0.1 formulas.

A higher expansion does not always raise the ceiling. With a one-million start, EXPANSION 4 stops at 21,000,000 items, and the default expansion of 2 reaches 63,000,000. The bigger jumps reach a sub-filter that cannot fit sooner.

127.0.0.1:6379> BF.RESERVE dedup:2026-11-10 0.001 5000000 NONSCALING
OK
127.0.0.1:6379> BF.RESERVE seen:urls 0.01 1000000 EXPANSION 2
OK
127.0.0.1:6379> BF.RESERVE bad 0.01 1000000 NONSCALING EXPANSION 2
(error) ERROR

Interpretation: the first command fixes the filter at five million items. The second scales from one million. The third fails because 1.0.1 accepts only one of the two options. The BF.RESERVE reference prose calls the option "SCALING <expansion rate>", but the keyword the module parses is EXPANSION.

How do you check that a filter can reach its target size?

VALIDATESCALETO n is an option of BF.INSERT, not BF.RESERVE. Before creating anything, valkey-bloom simulates the scale-outs for the capacity, rate, expansion and tightening on the command line. If n items cannot be reached under the current memory limit, or if the rate would degrade to zero first, the command is rejected and no key is created. It cannot be combined with NONSCALING, as the BF.INSERT reference states.

127.0.0.1:6379> BF.INSERT seen:events CAPACITY 1000000 ERROR 0.01 VALIDATESCALETO 63000001
(error) ERR provided VALIDATESCALETO causes bloom object to exceed memory limit
127.0.0.1:6379> BF.INSERT seen:events CAPACITY 1000000 ERROR 0.01 VALIDATESCALETO 63000000
(empty array)
127.0.0.1:6379> BF.INFO seen:events MAXSCALEDCAPACITY
(integer) 63000000

Interpretation: by the 1.0.1 formulas, the ceiling for a one-million, 1% filter is 63,000,000 items in six sub-filters, so one item more is refused. The validation runs on the arguments you pass, not on an existing key. For a filter that already exists, read BF.INFO key MAXSCALEDCAPACITY, which recomputes the ceiling under today's limit. The other refusal reads ERR provided VALIDATESCALETO causes false positive to degrade to 0.

How do large filters behave under replication and persistence?

Four behaviours in the 1.0.1 release notes and source matter once filters reach tens of megabytes.

• Replication is deterministic. When a primary creates a filter, it replicates a BF.INSERT carrying the exact capacity, rate, tightening, expansion and seed, so the replica builds an identical bitmap. Adds that create nothing are replicated verbatim. Replicas skip the memory-limit check on replicated commands.
• RDB loads do check the limit. The RDB loader recomputes each filter's size and refuses an object over the current bf.bloom-memory-usage-limit, logging Failed to restore bloom object: Object larger than the allowed memory limit. A replica or restore target with a lower limit than the primary can therefore fail its full sync or restore.
• Snapshots carry the whole bitmap. RDB saves every sub-filter's bitmap, and an AOF rewrite emits one BF.LOAD per filter with the encoded object. Budget RDB, AOF and full-sync transfer size at roughly the filter's Size.
• One filter is one key in one slot. In cluster mode, a 100 MB filter sits on one shard and moves as one key when its slot is migrated. Split by tenant or time window if you need it spread across shards.

# On each node: confirm the module version, the limit, and Bloom memory
valkey-cli MODULE LIST
valkey-cli CONFIG GET bf.bloom-memory-usage-limit
valkey-cli INFO BF
valkey-cli MEMORY USAGE seen:events

Interpretation: MODULE LIST shows the module as bf, with version 10001 for 1.0.1. INFO BF reports bf_bloom_total_memory_bytes, bf_bloom_num_objects and bf_bloom_num_filters_across_objects, plus defrag hit and miss counters. A rising ratio of sub-filters to objects means filters are scaling out more than you planned.

Is valkey-bloom compatible with RedisBloom commands?

For the core commands, yes. The valkey-bloom README says it is API compatible with the Bloom filter command syntax of the Valkey client libraries and their Redis equivalents. The differences are at the edges.

Because the type names and hashing differ, filters move by re-inserting the source items, not by copying RDB files. The Redis Stack to valkey-bundle migration guide covers that cutover. For the wider server comparison, see Redis vs Valkey.

Which patterns suit a Bloom filter: deduplication or a cache-penetration guard?

A false positive means something different in each, and Bloom filters cannot delete items, so pick the rate and key layout to match.

Deduplication

BF.ADD is check-and-set in one command: it returns 1 if the item was new and 0 if it was probably seen. Use the return value directly. A separate BF.EXISTS followed by BF.ADD leaves a race between two workers.

# One key per day, sized for the day's volume, expiring after a week
127.0.0.1:6379> BF.RESERVE dedup:2026-11-10 0.001 5000000 NONSCALING
OK
127.0.0.1:6379> EXPIRE dedup:2026-11-10 604800
(integer) 1
127.0.0.1:6379> BF.ADD dedup:2026-11-10 order:81234:email-sent
(integer) 1
127.0.0.1:6379> BF.ADD dedup:2026-11-10 order:81234:email-sent
(integer) 0

Interpretation: a 0 on a genuinely new item is a false positive, so that item is skipped. At 0.001, about one new item in a thousand is skipped once the day's filter is full. If a skipped notification or event is not acceptable, confirm a 0 against the source of truth before dropping the item. Time-windowed keys also give you deletion: the whole day expires together.

Cache-penetration guard

Requests for IDs that do not exist miss the cache every time and fall through to the database. A filter of every valid ID answers "definitely not" without a database read. The cache-aside and invalidation side of this design is covered in solving cache invalidation and stampede in Valkey.

import valkey

r = valkey.Valkey(host="10.0.0.5", port=6379)

def get_product(product_id: str):
    # 0 means the ID was never added: skip the cache and the database
    if r.execute_command("BF.EXISTS", "valid:products", product_id) == 0:
        return None
    cached = r.get(f"product:{product_id}")
    if cached is not None:
        return cached
    return load_from_db_and_cache(product_id)   # your existing path

def create_product(product_id: str, row):
    insert_into_db(row)
    r.execute_command("BF.ADD", "valid:products", product_id)

Interpretation: every write path that creates an ID must also add it to the filter, or new rows will look absent. Deleted IDs stay in the filter as permanent "maybe" answers, and they pass through to the database as before. Rebuild the filter on a schedule under a new key, then switch readers with RENAME. A guard sees mostly misses, which check every sub-filter, so reserve it as NONSCALING or with a large starting capacity.

Frequently asked questions

Key takeaways

Size and tune Valkey Bloom filters with JusDB

JusDB's Valkey performance tuning team can review your Bloom filter keys against their real growth. That covers capacity, error rate, sub-filter counts and the per-filter memory limit on every node. We can then help plan resized filters and the rebuild or cutover to move traffic onto them.

Bring the output of INFO BF, BF.INFO for your largest filters, and your expected item counts. Your team approves every production change.

Related reading

• Valkey performance tuning: memory, persistence, and cluster config
• Redis memory optimization: encodings, TTLs, and eviction policies
• Solving cache invalidation and stampede in Valkey with Debezium CDC
• Valkey documentation: Bloom filters