This page is the Simplified Technical English version of ZIP 222. The source ZIP is the normative text. Read the writing rules to understand the marked words.
ZIP 222: Transparent Zcash Extensions
ZIP: 222
Title: Transparent Zcash Extensions
Owners: Jack Grigg <thestr4d@gmail.com>
Kris Nuttycombe <kris@nutty.land>
Credits: Zaki Manian
Daira-Emma Hopwood
Sean Bowe
Status: Draft
Category: Consensus
Created: 2019-07-01
License: MIT
Discussions-To: <https://github.com/zcash/zips/issues/1231>
Terminology
On this page, STE words are in the place of the BCP 14 key words of the source. Refer to the key word table.
In this document, the term "network upgrade" has the meaning that is described in ZIP 200 [zip-0200].
In this document, the term "prefix-free" has this meaning. Take any value of a type. No valid encoding of this value can have the same binary representation as any prefix of the binary encoding of another value of this type.
In this document, the term "non-malleable" has the meaning that is described in ZIP 244 [zip-0244].
The value MAX_MONEY is defined in section 5.3 of the Zcash Protocol Specification [protocol-constants].
Abstract
This ZIP contains a change to the consensus rules. With this change, complex forms of transparent output preconditions can be deployed in network upgrades. Thus it becomes possible to create "transparent Zcash extensions". These extensions add to the functionality of the network in a carefully-defined and auditable way.
Motivation
Zcash supports a limited set of preconditions that can be imposed on how funds, both transparent and shielded, can be spent. The spending limitations on transparent funds are defined by what can be encoded in Bitcoin script. The spending of shielded funds is even more restricted. Thus Zcash does not yet support some use cases. An example is the integration of BOLT support [zip-draft-bolt].
Transparent Zcash Extensions are intended to make it possible to add new functionality in steps. The intention is to add this functionality with no change to the interpretation of the existing Bitcoin script. This script is complex and potentially error-prone. Extensions can also be laboratories to evaluate the demand for functionality. This functionality can eventually become a candidate for inclusion in the consensus rules for shielded transactions.
Definitions
- encumber
- To set conditions that must be satisfied to spend some or all of the outputs of a transaction.
- precondition
- A challenge that must be satisfied to gain spending authority over an encumbered amount.
- witness
- Evidence to be evaluated against the challenge that is encoded by a precondition.
Specification
Transparent Extensions
Transparent Extensions are modular software components that are distributed as part of the code of consensus implementations. An extension defines interpretation rules for several new pieces of data. A transaction includes these pieces of data.
A Transparent Extension is identified by a numeric type. A Transparent Extension can also have several modes of operation. These modes correspond to different kinds of encumbrance within the overall protocol of the extension.
These three values are available to the extension (in addition to type):
- A numeric
mode. - A byte sequence,
precondition, that contains an encoding of the precondition. This encoding is prefix-free within this(type, mode). - A byte sequence,
witness, that contains an encoding of evidence. The evidence purports to satisfy the precondition. This encoding must be prefix-free within this(type, mode).
The extension is responsible for the mode-specific parsing and serialization of these data fields. In addition, the extension must implement a deterministic verification algorithm, tze_verify. The arguments of tze_verify are:
modepreconditionwitness- a context object
The context object contains deterministic public information about the transaction and about the block chain (up to and including the block that the transaction is mined in). tze_verify returns true if the precondition is satisfied in that context. Otherwise it returns false.
An extension can request that arbitrary public information about the transaction and the block chain be included in the context object of the extension. A caller that integrates the extension at the integration point must satisfy these requirements.
We recommend that an extension restricts the information that it requests to the information that node implementations can supply in an efficient manner. For example, we recommend that an extension does not request full blocks to construct or validate a precondition. We also recommend that an extension requests only the transaction data that is essential for its computation.
In addition, an extension can request some preprocessing by the consensus-validating node. But we recommend that the construction of such contextual data does not have significant computational costs.
In a ZIP that contains the definition of a new transparent extension, these items must be completely specified:
- the structure of all three of the specified values for each defined mode
- the behavior of
tze_verify - any contextual information that is necessary
The encoded forms of precondition and witness do not have to be constant-length. But we recommend that these forms are solely determined by the pair (type, mode).
With the introduction of TZEs by this ZIP, there is a new transparent unspent outpoint set. This set is distinct from the UTXO set. Indices into this set of outpoints can be referred to by TZE inputs in spending transactions.
Encoding in transactions
We define a new transaction format. This format contains two new fields:
tze_outputs: A list of pairs. Each pair has two items:- The value that is encumbered.
- The precondition that must be satisfied to spend the value.
tze_inputs: A list of pairs. Each pair has two items:- An outpoint that references a prior precondition.
- A witness that satisfies it.
The transaction format must be non-malleable, in this sense. Any change to the effects of the transaction changes its transaction ID. But any valid change to a witness inside tze_inputs does not change the transaction ID. This will be specified in a separate ZIP.
A new version <TBD> transaction format and corresponding version group identifier <TBD> will be introduced in the same network upgrade (a hard fork) as TZE functionality. The version <TBD> format differs from the version 4 transaction format as follows. A length-prefixed encoding of TZE inputs and outputs is added to the serialized transaction format. It is immediately after the fields for transparent inputs and outputs.
| Version | Field | Description | Type |
|---|---|---|---|
... | ... as before | ... | ... |
| >= 1 | tx_in_count | variable-length integer | compactSize |
| >= 1 | tx_in | list of inputs | vector |
| >= 1 | tx_out_count | variable-length integer | compactSize |
| >= 1 | tx_out | list of outputs | vector |
| >= <TBD> | tze_in_count | variable-length integer | compactSize |
| >= <TBD> | tze_in | list of TZE inputs | vector |
| >= <TBD> | tze_out_count | variable-length integer | compactSize |
| >= <TBD> | tze_out | list of TZE outputs | vector |
| >= 1 | lock_time | block height or timestamp | uint32 |
... | ... as before | ... | ... |
The common serialized form tze_data is used in both the tze_in vector and the tze_out vector. This form is described below. The serialization of all integer and vector types is the same as in Bitcoin.
tze_data encoding:
| Field | Description | Type |
|---|---|---|
tze_id | extension type | compactSize |
tze_mode | extension mode | compactSize |
tze_data_payload_len | length of precondition/witness data | compactSize |
tze_data_payload | serialized precondition/witness data | tze_data_payload_len bytes |
TZE Input Encoding:
| Field | Description | Type |
|---|---|---|
prevout_hash | previous txid | uint256 |
prevout_in | index into previous txn's outputs | uint32 |
witness | witness for prevout's precondition | tze_data |
TZE Output Encoding:
| Field | Description | Type |
|---|---|---|
amount | spendable amount, in zatoshi | int64 |
precondition | encodes a precondition encumbering amount | tze_data |
Consensus rules
Once this ZIP becomes active, these new consensus rules are enforced:
- For each
(outpoint, witness)pair intze_inputs:outpointmust reference a precondition of the same type and mode in an already-mined transaction.tze_verify(mode, precondition, witness, context)must returntrue.
- If a transaction has non-empty
tze_inputsand non-emptytze_outputs, every element in both fields must have the sametype. The purpose is to eliminate the possibility of cross-extension attacks. Such attacks are not a consideration if onlytze_inputsor onlytze_outputsare present. Thus in that case, the extensiontypecan vary between elements. - Non-coinbase transactions must have at least one of these items:
- nonempty transparent inputs
- nonempty shielded inputs
- nonempty
tze_inputs
In [protocol-txnconsensus], the text [Sapling onward] At least one of tx_in_count,
nShieldedSpend, and nJoinSplit MUST be nonzero is replaced by the rule above.
- Transactions must have at least one of these items:
- nonempty transparent outputs
- nonempty shielded outputs
- nonempty
tze_outputs
- All
amountfield values oftze_outputrecords must be nonnegative and must not be greater thanMAX_MONEY. - For a transaction, the sum of amounts that leave the transparent value pool must not be greater than the sum of amounts that enter it.
- The amounts that leave the pool are the Bitcoin-style outputs and the TZE outputs, plus the JoinSplit
vpub_oldvalues. - The amounts that enter the pool are the Bitcoin-style inputs and the TZE inputs, plus the JoinSplit
vpub_newvalues, plus the SaplingvalueBalanceamount.
- The amounts that leave the pool are the Bitcoin-style outputs and the TZE outputs, plus the JoinSplit
Changes to signatures over transaction digests
This ZIP must be deployed in conjunction with or after ZIP 244 [zip-0244]. New non-malleable transaction identifier and signature digest algorithms are defined in ZIP 244.
The transaction digests for transaction identifiers and signatures will include the newly added parts of the transaction, but not the witness information. That is, they will not include the witness fields of TZE Input Encodings. Refer to ZIP 245 [zip-0245] for the specification of these new digests. If the changes in this ZIP are deployed, the changes that are described in ZIP 245 must be deployed also.
Rationale
A single extension type must be used in each transaction that has both TZE inputs and TZE outputs, to prevent cross-protocol attacks. The disadvantage is that, with this restriction, no TZE-encumbered value can be spent directly into a different TZE type. The value must go through a regular address in between. This restriction can be relaxed in future ZIPs for specific combinations of (type, mode) pairs that were analyzed for cross-protocol attacks. But here we select a fail-safe default behaviour.
A transaction that has TZE inputs and no TZE outputs is not subject to single-extension restrictions or single-mode restrictions. Likewise, a transaction that has TZE outputs and no TZE inputs can have TZE outputs for multiple extension-type/mode pairs. This is because the potential for cross-protocol attacks in this situation is negligible.
In an earlier draft version of this ZIP, the payloads were stored inside transparent inputs and outputs. This had the advantage that no change to the transaction format was necessary. But the consensus rules were significantly more complicated. Also, in that design, the extension logic was coupled too tightly to the transparent address logic. Instead, dedicated transaction fields and a separate unspent output set are used in this ZIP.
Security and Privacy Considerations for Future TZE Implementations
This ZIP is based on the assumption that the base transaction format is non-malleable. But the precondition and witness byte sequences are treated here as opaque. It is the responsibility of tze_verify to enforce these items:
preconditionmust be non-malleable. Any malleation must causetze_verifyto returnfalse.- The output of
tze_verify(mode, precondition, witness, context)must be deterministic.
A ZIP that contains the definition of a new extension type must include a section about how any potential sources of malleability are handled.
This ZIP includes restrictions to prevent cross-protocol attacks. But the extension mode is another potential attack surface. It is the responsibility of ZIPs that contain definitions of new extensions to include one or both of these items:
- An examination of the potential for cross-mode attacks, in their security analysis.
- An appropriate restriction on the modes that can be combined in a single transaction.
Reference Implementation
- Librustzcash reference implementation of the TZE API: [librustzcash_zip222]
- Zcashd reference implementation of the consensus rule changes: [zcashd_zip222]
Acknowledgements
Zaki Manian suggested the handler semantics of tze_verify. These semantics are based on the design of Cosmos. Daira-Emma Hopwood and Sean Bowe gave useful feedback on an early draft of this ZIP. They also helped to analyse the various sources of transaction ID malleability.
We also thank the many other people who took part in discussions at Zcon1. These discussions led to the earlier draft version of this ZIP.
References
- [BCP14]
- Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"
- [protocol]
- Zcash Protocol Specification, Version 2021.2.16 or later
- [protocol-constants]
- Zcash Protocol Specification, Version 2021.2.16. Section 5.3: Constants
- [protocol-txnconsensus]
- Zcash Protocol Specification, Version 2021.2.16. Section 7.1: Transaction Consensus Rules
- [zip-0032]
- ZIP 32: Shielded Hierarchical Deterministic Wallets
- [zip-0200]
- ZIP 200: Network Upgrade Mechanism
- [zip-0244]
- ZIP 244: Transaction Non-Malleability Support
- [zip-0245]
- ZIP 245: Transaction Identifier Digests & Signature Validation for Transparent Zcash Extensions
- [zip-draft-bolt]
- Draft ZIP: Add support for Blind Off-chain Lightweight Transactions (Bolt) protocol
- [librustzcash_zip222]
- Rust language reference implementation of TZE API
- [zcashd_zip222]
- zcashd reference implementation of consensus rule changes