Compact Block Filters

Anna opens her phone wallet after some time away. It needs to find out whether anyone has paid her since its last connection, or whether any of her funds have been spent. Instead of all transactions, it downloads small filters and compares them with the output conditions it watches. An output is the part of a transaction specifying an amount and the conditions for spending it next. On a match, the application requests the whole block and looks for an actual transaction. It must not increase the balance from a filter match alone. The filter is neither an account statement nor proof of ownership. Before searching, the client synchronises block headers—brief identifying block data—and the linked filter headers. [1]

With the older BIP 37 approach, the client sent a peer a filter built from the data it wanted to watch. Here the direction is reversed: the server supplies a block filter and Anna searches locally. Clients receive the same contents for the same block and filter type. They need not directly reveal their search scripts to each server. This does not provide anonymity: a peer may still see an IP address, connection timing and which full blocks the wallet subsequently requests. Spreading downloads across different peers reduces the information gathered by any one of them, but does not remove all ways to correlate it. [1]

Compact Block Filters select blocks worth examining. Compact Blocks under BIP 152 solve another problem: saving bandwidth when relaying a block by using short identifiers for transactions the receiver often already knows. It draws these from its Mempool, its local collection of unconfirmed transactions, and requests any missing ones. Neither tool alone checks every Bitcoin rule. A Full Node independently validates blocks and transactions against the network’s rules. A light wallet checking only headers and the inclusion of its transaction performs a narrower check. Saving data and fully validating it are different properties. [1] [5]

BIP 158 specifies the format. The basic filter includes scriptPubKey, the script defining spending conditions, from new outputs except those beginning with OP_RETURN. It also includes the scripts of earlier outputs spent within the block. For inputs, the exception is coinbase, the block’s first transaction distributing the miner’s reward, which spends no ordinary previous outputs. Its new outputs follow the same inclusion rules as other new outputs. Empty items are omitted. A wallet must convert an address into the correct script; searching for its printed text is insufficient. The filter is not a general search engine for txid transaction identifiers, witness data such as signatures, or arbitrary notes. [2] [3]

After downloading a block, Anna may find that nothing in it concerns her. Such a false match is a normal trade-off for a small filter, not by itself evidence of a fault. Technically, items are mapped to numerical fingerprints, sorted, and the differences stored compactly using Golomb-Rice coding. For basic, P=19 and M=784931. A single query for an absent item has a match probability of approximately 1/M; watching more items creates more chances for a match. This is not a fixed error rate for an entire wallet. A correctly constructed filter does not omit an included item. A negative result is reliable only with a correct filter, a covered data type and the right query. [2]

BIP 157 defines filter transport and their linked headers. Each header connects the hash—a cryptographic fingerprint—of the current filter with the previous header. Changing a filter breaks the expected link. However, this chain is not part of the Bitcoin block header, and mining does not directly commit to its contents. A dishonest peer can prepare false filters together with correctly calculated linked fingerprints. It does not change Bitcoin, but can conceal information from an isolated client. Checking the mathematical linkage must therefore be accompanied by checking that the underlying data is correct. [1]

BIP 157 relies on comparing peers with at least one honest source. If they disagree, the client can locate the first divergence and verify the filter against underlying data. Building a basic filter may require more than the new block alone: it also needs the scripts of earlier outputs that its transactions spend. A reorganisation needs special handling: the network switches to another branch of valid blocks with greater cumulative proof of work. The wallet must connect filters to the new branch and update its history and balance. The same height, meaning the same position in the chain, need not identify the same block. [1] [3]

A wallet needs available filters and a way to obtain selected full blocks. A node may delete old blocks after constructing their filters, leaving another source to supply them. In Bitcoin Core, getblockfilter returns a filter and its header for a specified block when the appropriate index is enabled. That command’s availability does not establish public P2P service to other nodes or access to the full history. Filters of existing blocks also do not announce unconfirmed transactions in the Mempool. Showing a pending payment needs another mechanism. These operational details determine whether a light wallet actually works well. [1] [4]