As Bitcoin continues to make gains in adoption around the world, more and more new and existing users are concerned about their privacy in this new financial framework. We’ve come to expect from legacy financial systems that employers, customers, and the people we trade with will have limited knowledge of our finances appropriate to our relationships with them. However, in the midst of global spy networks and corporate collusion in the forces undermining online confidentiality, Bitcoiners are understandably cautious about the open-book nature of the blockchain. There are a few user habits and upcoming technologies that can greatly enhance users’ control over whom they share their financial details with when using Bitcoin, maintaining the pseudonymity that it provides.
The single most effective tactic for preventing privacy leaks into the blockchain is also the oldest advice that's ever been offered on the subject - each receiving address should only be used to receive bitcoins once. If addresses are never reused, and if Bitcoin clients randomize the order of the outputs, then the transaction graph is not easy to trace. Done properly, the blockchain would be kept free of privacy-reducing transaction.
The goal of eliminating address reuse can be achieved with carefully-coded client, except for one case. It's possible for a user to end up in a situation where he or she wants to spend more bitcoins than are present in any of their addresses. The only way to do this is to create a transaction with more than one input (from different addresses). There is nothing wrong with kind of transaction from a protocol perspective, however it's catastrophic for privacy. Combining inputs from multiple addresses is strong evidence that the same entity controls both relevant private keys and this evidence survives in the public blockchain forever. At the present time no wallet software in use warns the user when they create a transaction the permanently damages their financial privacy.
The only circumstance in which a multiple-input transaction is not a privacy disaster would be a hypothetical network in which CoinJoin transactions are common, and where regular spends can be constructed to look indistinguishable from CoinJoin transactions and where all these transactions do not leak any information about the correlation between inputs and outputs. It's not yet apparent that all of these are possible to achieve.
The most reliable way to avoid input merging is to use multiple transactions when no sufficiently large output is available. If our user has one address with 2 BTC and another address with 3 BTC and wants to spend 4 BTC, then ideally he’d obtain two addresses for the recipient and create two independent transactions. This allows the user to avoid output merging without leaving any trace in the blockchain to correlate the two transactions. The only way for an attacker to deduce a relationship between them would be to attempt to identify the origin node of all transactions. This is relatively straightforward to combat by smart broadcasting techniques and will become naturally more difficult over time as the transaction volume of Bitcoin grows.
The only missing piece for this to happen is a method for the recipient to always be able to obtain as many deposit addresses as necessary. This requires cooperation from the recipient, as well as protocols and best practices education to achieve. We have the perfect opportunity to achieve this in the form of integration of BIP32 (Hierarchical Deterministic Wallet) keys into the soon-to-be-released payment protocol.
As I’ve written before, the extended public key (xpub) defined by BIP32 is an ideal replacement for Bitcoin addresses from a privacy perspective. If one user gives another user an extended public key, that user can generate an arbitrarily large number of deposit addresses with perfect assurance the recipient will be able to spend the resulting transactions. Even better, this key exchange generally only needs to happen once between any two entities instead of every time they interact. This reduces an attacker’s opportunity to intercept and alter the deposit address to only the first key exchange.
The present version of the payment protocol does not incorporate BIP32. It does include a method for the sender to request more than one deposit address, however since this is optional on the part of the recipient it gives control of the sender’s privacy to the recipient. This inversion of control is not something we should build into the long-term infrastructure of the network.
The only missing piece for this to happen is a method for the recipient to always be able to obtain as many deposit addresses as necessary. This requires cooperation from the recipient, as well as protocols and best practices education to achieve. We have the perfect opportunity to achieve this in the form of integration of BIP32 (Hierarchical Deterministic Wallet) keys into the soon-to-be-released payment protocol.
As I’ve written before, the extended public key (xpub) defined by BIP32 is an ideal replacement for Bitcoin addresses from a privacy perspective. If one user gives another user an extended public key, that user can generate an arbitrarily large number of deposit addresses with perfect assurance the recipient will be able to spend the resulting transactions. Even better, this key exchange generally only needs to happen once between any two entities instead of every time they interact. This reduces an attacker’s opportunity to intercept and alter the deposit address to only the first key exchange.
The present version of the payment protocol does not incorporate BIP32. It does include a method for the sender to request more than one deposit address, however since this is optional on the part of the recipient it gives control of the sender’s privacy to the recipient. This inversion of control is not something we should build into the long-term infrastructure of the network.
Discussions regarding the payment protocol have revealed several objections to allowing customers to generate their own deposit addresses on demand. This article will examine the two most common and relevant objections.
Ease of Accounting
“For every complex problem, there is a solution that is simple, neat, and wrong.”
Integrating Bitcoin into the workflow of an existing business is not at all straightforward. Unlike transactions in the legacy financial system, there is no such thing as an account number. Nor is there any concept of a sending account to which refunds may be sent. Many people who have created merchant tools try to emulate these concepts by careful use of addresses. These attempts vary from fragile to completely broken.
The most common tactic is to connect payments to invoices by generating a new address for each invoice. This is easy to code, and makes intuitive sense, and is what most merchants using the payment protocol will probably do by default. Unfortunately, it’s subtly broken from a business accounting perspective because the assumption of a 1:1 relationship between invoices and payments is one that frequently breaks in the real world.
The most common tactic is to connect payments to invoices by generating a new address for each invoice. This is easy to code, and makes intuitive sense, and is what most merchants using the payment protocol will probably do by default. Unfortunately, it’s subtly broken from a business accounting perspective because the assumption of a 1:1 relationship between invoices and payments is one that frequently breaks in the real world.
Imagine a customer who signed up for mobile phone service in January at a $50 monthly rate. Everything is fine until March, where she is late paying due to unforeseen circumstances. She’s not able to make a payment until after the April bill has already posted, at which time she pays $75. One week later, she pays the remaining $25 on the April bill and is caught back up.
Situations like arise frequently in B2C (business to customer) and B2B (business to business) relationships. What would happen if the database storing the payment and invoice records assumed a 1:1: relationship between them? There would be no way to reconcile what actually happened. Understanding how the phone company’s account system handles this situation is the key to understanding how Bitcoin merchants should design their payment interfaces.
In order to handle unexpected payment scenarios, business accounting systems use the concept of net billing. Payments are issued in order, and invoices are received in order, so it’s possible to keep running totals for both. Invoices are considered paid when a payment is received that pushes the running total of payments above the running total for the invoice. This concept is illustrated in the table below:
Situations like arise frequently in B2C (business to customer) and B2B (business to business) relationships. What would happen if the database storing the payment and invoice records assumed a 1:1: relationship between them? There would be no way to reconcile what actually happened. Understanding how the phone company’s account system handles this situation is the key to understanding how Bitcoin merchants should design their payment interfaces.
In order to handle unexpected payment scenarios, business accounting systems use the concept of net billing. Payments are issued in order, and invoices are received in order, so it’s possible to keep running totals for both. Invoices are considered paid when a payment is received that pushes the running total of payments above the running total for the invoice. This concept is illustrated in the table below:
Invoice | Payment | ||||||
Number | Date | Amount | Total | Number | Date | Amount | Total |
1 | 2013-01-31 | $50 | $50 | 1 | 2013-02-15 | $50 | $50 |
2 | 2013-02-28 | $50 | $100 | 2 | 2013-03-15 | $50 | $100 |
3 | 2013-03-31 | $50 | $150 | 3 | 2013-05-07 | $75 | $175 |
4 | 2013-04-30 | $50 | $200 | 4 | 2013-05-15 | $25 | $200 |
For the first two months, there was indeed exactly one payment for each invoice. This pattern was broken in the March billing cycle, but the running totals make it easy to see that invoice #3 was paid in full by payment #3, and invoice #4 was paid by a combination of payments #3 and #4.
Any accounting system which must interface with real people must incorporate net billing, or else it will sooner or later fail spectacularly. Any accounting system in use by a Bitcoin merchant which can not easily handle the case of a single invoice being paid with multiple transactions is broken anyway therefore there is no reason to restrict a single invoice to one Bitcoin address.
Ease of accounting is not a valid reason to refrain from integrating xpubs into the payment protocol.
Any accounting system which must interface with real people must incorporate net billing, or else it will sooner or later fail spectacularly. Any accounting system in use by a Bitcoin merchant which can not easily handle the case of a single invoice being paid with multiple transactions is broken anyway therefore there is no reason to restrict a single invoice to one Bitcoin address.
Ease of accounting is not a valid reason to refrain from integrating xpubs into the payment protocol.
Malicious customer behavior
There are several ways customers could misbehave and create undue hardship for a merchant if they are given the ability to create an arbitrarily-large number of deposit addresses. All of these attacks are equivalent to attacks which can occur when they are only given a single deposit address therefore not a reason to discriminate against an xpub-based payment protocol.
The most obvious way to misbehave is to create extra nodes in the chain of addresses generated by an xpub and/or generate individual address out of sequence. A customer who did this could generate a valid address which the merchant could spend, but their wallet would not recognize the payment because it can’t scan the entire keyspace to locate every possible address a single xpub can generate.
This attack can be mitigated by adhering to the expectation that addresses will be created in numerical index order, with no branching, and deviating from this expectation is considered non-standard behavior.
The most obvious way to misbehave is to create extra nodes in the chain of addresses generated by an xpub and/or generate individual address out of sequence. A customer who did this could generate a valid address which the merchant could spend, but their wallet would not recognize the payment because it can’t scan the entire keyspace to locate every possible address a single xpub can generate.
This attack can be mitigated by adhering to the expectation that addresses will be created in numerical index order, with no branching, and deviating from this expectation is considered non-standard behavior.
To avoid accidental cases of out-of-order address generation or reuse, the merchant can send an index hint along with the payment request. Perhaps a merchant generates one xpub per customer, and a customer has ordered previously. If the first 20 addresses have already been used, the merchant can send an index hint of 21 with the next invoice, in case the customer is paying from a different client that does not have access to the previous transaction history. Any remaining cases which slip through can be dealt with via the address lookahead window which any deterministic wallet must implement anyway.
Malicious construction of non-standard BIP32 sequences is no different than malicious construction of non-standard Bitcoin transactions using a single address. A customer could also create an output that is different from a standard pay to sighash script which would be spendable by the merchant but which their wallet would not recognize as belonging to them.
Malicious construction of non-standard BIP32 sequences is no different than malicious construction of non-standard Bitcoin transactions using a single address. A customer could also create an output that is different from a standard pay to sighash script which would be spendable by the merchant but which their wallet would not recognize as belonging to them.
Another possible attack would be to generate an unreasonable amount of addresses and transactions to complete payment with. Perhaps a disgruntled customer may decide to pay a 10 BTC invoice with 10000 individual 1 µBTC outputs each sent to a different address. This would indeed be a burden on the merchant however it’s possible to conduct an equivalent attack by sending all 10000 outputs to the same address.
For every case of pathological user behavior enabled by giving them an extended public key there is an equivalent attack that’s possible now when customers are only given single addresses.
For every case of pathological user behavior enabled by giving them an extended public key there is an equivalent attack that’s possible now when customers are only given single addresses.
Malicious customer behavior is not a valid reason to refrain from integrating xpubs into the payment protocol.
The next 18 to 24 months are crucial to Bitcoin’s long term future. If the history of similar software projects is any indication, the protocols which are being built today will be around for longer than anyone expects. It is for this reason that long-term privacy must not be sacrificed for short term ease of implementation. The payment protocol should not default to denying customers the ability to retain their financial privacy. Building BIP32 extended public keys into the protocol and requiring merchant tools to be compatible with net billing is the right thing to do for both accounting purposes, and for preserving the financial privacy of both customers and merchants alike.