BIP 157 Client-Side Block Filtering: Compact Filters Explained

BIP 157 is the peer-to-peer protocol that lets a light Bitcoin wallet find its transactions without telling the network which addresses it owns. Full nodes compute a compact filter for every block, defined in the companion BIP 158; the wallet downloads the filters, tests them against its own scripts on the device and fetches a full block only when a filter matches. It was designed as an alternative to the Bloom filters of BIP 37, which leaked a wallet’s addresses to the nodes serving it.

BIP 157 at a Glance

Why Did Light Wallets Need BIP 157?

A light wallet does not download and check every block. It follows the chain of block headers, which are only 80 bytes each, and asks full nodes for the data that concerns it. Before BIP 157, the standard way to ask was BIP 37: the wallet sent a Bloom filter describing its addresses, and the node returned matching transactions with proofs that they were in a block. The BIP 157 specification lists three problems with that design:

• Privacy. Research cited in the spec found that most implementations gave wallets “virtually zero privacy”: the filter told the node which addresses were the user’s.
• Omissions. A malicious node could leave out transactions with little risk of being caught, a serious problem for Lightning clients that must react to on-chain events.
• Load on honest nodes. Serving each client’s filter cost the node disk reads and CPU time, and crafted filters turned that into a denial-of-service attack.

Bitcoin Core switched BIP 37 support off by default in version 0.19, citing those denial-of-service vectors. BIP 157 turns the design around: the node builds one filter per block for everyone, and the wallet does the matching itself.

How Do Compact Block Filters Work?

BIP 158 defines the basic filter that nodes serve. For every transaction in a block it holds two kinds of items: the script each input spends (except in the coinbase transaction) and the script of each output, apart from OP_RETURN outputs. Every Bitcoin address stands for such a script, and a wallet can list all of its own from its extended public key, so it can test that list against the filter.

The items are hashed and stored as a Golomb-coded set, a sorted list of hash differences compressed with Golomb-Rice coding, with the parameters P = 19 and M = 784,931. The set always matches the items it contains; any other script matches with a probability of 1 in 784,931, a false positive that costs one needless block download.

A light client using BIP 157 goes through these steps:

1. Sync the block headers and follow the chain with the most proof of work.
2. Download the filter headers from several peers and compare them.
3. Download the filters for the blocks it needs and check each against its filter header.
4. Test the wallet’s scripts against each filter locally.
5. For a match, download the full block, preferably from a random peer, and pick out the wallet’s transactions.

How Does a Wallet Know a Filter Is Honest?

Filters are not part of Bitcoin’s consensus rules, so a block header does not commit to them. BIP 157 adds its own chain of commitments instead: each filter header is the double SHA-256 of the filter’s hash joined with the previous filter header, much as each block header commits to the one before it. Nodes also answer requests for checkpoints, the filter headers at every 1,000th block, so a client can fetch ranges from several peers in parallel and check each range against the checkpoints.

If two peers send different filter headers, the client finds the first block where they disagree, downloads that block, computes the correct filter itself and bans the peer that lied. The guarantee therefore holds as long as the wallet is connected to at least one honest peer that serves filters. Committing to filters inside blocks would remove that assumption, but it would need a change to the consensus rules, which the authors chose to avoid by working purely at the peer-to-peer layer.

BIP 157 vs BIP 37 Bloom Filters

The bandwidth row is the real trade-off. Bitcoin Optech notes that the protocol uses significantly more bandwidth in the normal case than Bloom filters or address-index servers, in exchange for more privacy against spy nodes and less risk for honest ones (Optech topic page).

Which Wallets and Nodes Use BIP 157?

• Bitcoin Core can index BIP 158 filters since version 0.19 and serve them to peers since 0.21, as its list of implemented BIPs records. Serving needs both -blockfilterindex=1 and -peerblockfilters=1, according to the 0.21.0 release notes, and such nodes signal it with the NODE_COMPACT_FILTERS service bit that BIP 158 defines.
• Neutrino, a Go library from Lightning Labs, is a light client built around compact block filters for mobile Lightning wallets. The protocol itself is sometimes called Neutrino, which Optech points out is not quite correct.
• lnd lists neutrino as one of its three chain back-ends, next to bitcoind and btcd, so a Lightning node can run without a full node of its own.
• Wasabi Wallet checks BIP-158 filters on the user’s computer and, when one matches, downloads the whole block from a random peer over Tor, as its FAQ describes.

A hardware wallet does not use filters itself: it only signs, and the app it connects to does the syncing, whether through filters, a server or a full node.

What Are the Limits of BIP 157?

• Bandwidth. A wallet downloads a filter for every block it scans, plus whole blocks for matches and false positives.
• Some privacy leaks. The node that serves a block sees that the wallet wanted it. The specification recommends fetching blocks from random outbound peers, and Tor hides the wallet’s IP address.
• At least one honest peer. Filter headers expose a lying peer only if the wallet also talks to an honest one.
• Confirmed transactions only. Filters describe blocks, so an incoming payment still waiting in the mempool shows up only once a block includes it, unless the wallet watches the mempool some other way.
• Light-client security. As the specification puts it, light clients do not check the validity of every block in the chain with the most work and rely on miner incentives instead; a full node checks every rule itself. Our guide to the longest chain rule explains what following the most work means.

Frequently Asked Questions

Is BIP 157 the same as Neutrino?

No. BIP 157 and BIP 158 are the protocol and the filter format; Neutrino is one light client that implements them.

Do compact block filters need a soft fork?

No. BIP 157 works entirely at the peer-to-peer layer, and nodes opt in. A commitment to filters in blocks would need one, which is why BIP 158 leaves OP_RETURN outputs out of the filter: a future soft fork could put the commitment in an OP_RETURN output without a circular dependency.

How do I serve compact block filters from my node?

On Bitcoin Core 0.21 or later, add blockfilterindex=1 and peerblockfilters=1 to bitcoin.conf and restart. Building the index takes time and disk space; the specification lets a node prune old blocks once their filters are stored.

Does BIP 157 work with every address type?

Yes. The basic filter covers output scripts of every type, so legacy, SegWit and Taproot addresses all match the same way.

Explore the BIP series: What are Bitcoin Improvement Proposals? · BIP-32: Hierarchical Deterministic Wallets · BIP-39: The Seed Phrase Standard · BIP-44: Multi-Account Hierarchy · BIP-110 (BIP-444): Reduced Data Temporary Softfork · BIP39 vs BIP32 vs BIP44 compared