Zcash in Simplified Technical English

This page is the Simplified Technical English version of ZIP 222. The source ZIP is the normative text. Read the writing rules to understand the marked words.

ZIP 222: Transparent Zcash Extensions

ZIP: 222
Title: Transparent Zcash Extensions
Owners: Jack Grigg <thestr4d@gmail.com>
        Kris Nuttycombe <kris@nutty.land>
Credits: Zaki Manian
         Daira-Emma Hopwood
         Sean Bowe
Status: Draft
Category: Consensus
Created: 2019-07-01
License: MIT
Discussions-To: <https://github.com/zcash/zips/issues/1231>

Terminology

On this page, STE words are in the place of the BCP 14 key words of the source. Refer to the key word table.

In this document, the term "network upgrade" has the meaning that is described in ZIP 200 [zip-0200].

In this document, the term "prefix-free" has this meaning. Take any value of a type. No valid encoding of this value can have the same binary representation as any prefix of the binary encoding of another value of this type.

In this document, the term "non-malleable" has the meaning that is described in ZIP 244 [zip-0244].

The value MAX_MONEY is defined in section 5.3 of the Zcash Protocol Specification [protocol-constants].

Abstract

This ZIP contains a change to the consensus rules. With this change, complex forms of transparent output preconditions can be deployed in network upgrades. Thus it becomes possible to create "transparent Zcash extensions". These extensions add to the functionality of the network in a carefully-defined and auditable way.

Motivation

Zcash supports a limited set of preconditions that can be imposed on how funds, both transparent and shielded, can be spent. The spending limitations on transparent funds are defined by what can be encoded in Bitcoin script. The spending of shielded funds is even more restricted. Thus Zcash does not yet support some use cases. An example is the integration of BOLT support [zip-draft-bolt].

Transparent Zcash Extensions are intended to make it possible to add new functionality in steps. The intention is to add this functionality with no change to the interpretation of the existing Bitcoin script. This script is complex and potentially error-prone. Extensions can also be laboratories to evaluate the demand for functionality. This functionality can eventually become a candidate for inclusion in the consensus rules for shielded transactions.

Definitions

encumber
To set conditions that must be satisfied to spend some or all of the outputs of a transaction.
precondition
A challenge that must be satisfied to gain spending authority over an encumbered amount.
witness
Evidence to be evaluated against the challenge that is encoded by a precondition.

Specification

Transparent Extensions

Transparent Extensions are modular software components that are distributed as part of the code of consensus implementations. An extension defines interpretation rules for several new pieces of data. A transaction includes these pieces of data.

A Transparent Extension is identified by a numeric type. A Transparent Extension can also have several modes of operation. These modes correspond to different kinds of encumbrance within the overall protocol of the extension.

These three values are available to the extension (in addition to type):

  • A numeric mode.
  • A byte sequence, precondition, that contains an encoding of the precondition. This encoding is prefix-free within this (type, mode).
  • A byte sequence, witness, that contains an encoding of evidence. The evidence purports to satisfy the precondition. This encoding must be prefix-free within this (type, mode).

The extension is responsible for the mode-specific parsing and serialization of these data fields. In addition, the extension must implement a deterministic verification algorithm, tze_verify. The arguments of tze_verify are:

  • mode
  • precondition
  • witness
  • a context object

The context object contains deterministic public information about the transaction and about the block chain (up to and including the block that the transaction is mined in). tze_verify returns true if the precondition is satisfied in that context. Otherwise it returns false.

An extension can request that arbitrary public information about the transaction and the block chain be included in the context object of the extension. A caller that integrates the extension at the integration point must satisfy these requirements.

We recommend that an extension restricts the information that it requests to the information that node implementations can supply in an efficient manner. For example, we recommend that an extension does not request full blocks to construct or validate a precondition. We also recommend that an extension requests only the transaction data that is essential for its computation.

In addition, an extension can request some preprocessing by the consensus-validating node. But we recommend that the construction of such contextual data does not have significant computational costs.

In a ZIP that contains the definition of a new transparent extension, these items must be completely specified:

  • the structure of all three of the specified values for each defined mode
  • the behavior of tze_verify
  • any contextual information that is necessary

The encoded forms of precondition and witness do not have to be constant-length. But we recommend that these forms are solely determined by the pair (type, mode).

With the introduction of TZEs by this ZIP, there is a new transparent unspent outpoint set. This set is distinct from the UTXO set. Indices into this set of outpoints can be referred to by TZE inputs in spending transactions.

Encoding in transactions

We define a new transaction format. This format contains two new fields:

  • tze_outputs: A list of pairs. Each pair has two items:
    • The value that is encumbered.
    • The precondition that must be satisfied to spend the value.
  • tze_inputs: A list of pairs. Each pair has two items:
    • An outpoint that references a prior precondition.
    • A witness that satisfies it.

The transaction format must be non-malleable, in this sense. Any change to the effects of the transaction changes its transaction ID. But any valid change to a witness inside tze_inputs does not change the transaction ID. This will be specified in a separate ZIP.

A new version <TBD> transaction format and corresponding version group identifier <TBD> will be introduced in the same network upgrade (a hard fork) as TZE functionality. The version <TBD> format differs from the version 4 transaction format as follows. A length-prefixed encoding of TZE inputs and outputs is added to the serialized transaction format. It is immediately after the fields for transparent inputs and outputs.

VersionFieldDescriptionType
...... as before......
>= 1tx_in_countvariable-length integercompactSize
>= 1tx_inlist of inputsvector
>= 1tx_out_countvariable-length integercompactSize
>= 1tx_outlist of outputsvector
>= <TBD>tze_in_countvariable-length integercompactSize
>= <TBD>tze_inlist of TZE inputsvector
>= <TBD>tze_out_countvariable-length integercompactSize
>= <TBD>tze_outlist of TZE outputsvector
>= 1lock_timeblock height or timestampuint32
...... as before......

The common serialized form tze_data is used in both the tze_in vector and the tze_out vector. This form is described below. The serialization of all integer and vector types is the same as in Bitcoin.

tze_data encoding:

FieldDescriptionType
tze_idextension typecompactSize
tze_modeextension modecompactSize
tze_data_payload_lenlength of precondition/witness datacompactSize
tze_data_payloadserialized precondition/witness datatze_data_payload_len bytes

TZE Input Encoding:

FieldDescriptionType
prevout_hashprevious txiduint256
prevout_inindex into previous txn's outputsuint32
witnesswitness for prevout's preconditiontze_data

TZE Output Encoding:

FieldDescriptionType
amountspendable amount, in zatoshiint64
preconditionencodes a precondition encumbering amounttze_data

Consensus rules

Once this ZIP becomes active, these new consensus rules are enforced:

  • For each (outpoint, witness) pair in tze_inputs:
    • outpoint must reference a precondition of the same type and mode in an already-mined transaction.
    • tze_verify(mode, precondition, witness, context) must return true.
  • If a transaction has non-empty tze_inputs and non-empty tze_outputs, every element in both fields must have the same type. The purpose is to eliminate the possibility of cross-extension attacks. Such attacks are not a consideration if only tze_inputs or only tze_outputs are present. Thus in that case, the extension type can vary between elements.
  • Non-coinbase transactions must have at least one of these items:
    • nonempty transparent inputs
    • nonempty shielded inputs
    • nonempty tze_inputs

In [protocol-txnconsensus], the text [Sapling onward] At least one of tx_in_count, nShieldedSpend, and nJoinSplit MUST be nonzero is replaced by the rule above.

  • Transactions must have at least one of these items:
    • nonempty transparent outputs
    • nonempty shielded outputs
    • nonempty tze_outputs
  • All amount field values of tze_output records must be nonnegative and must not be greater than MAX_MONEY.
  • For a transaction, the sum of amounts that leave the transparent value pool must not be greater than the sum of amounts that enter it.
    • The amounts that leave the pool are the Bitcoin-style outputs and the TZE outputs, plus the JoinSplit vpub_old values.
    • The amounts that enter the pool are the Bitcoin-style inputs and the TZE inputs, plus the JoinSplit vpub_new values, plus the Sapling valueBalance amount.

Changes to signatures over transaction digests

This ZIP must be deployed in conjunction with or after ZIP 244 [zip-0244]. New non-malleable transaction identifier and signature digest algorithms are defined in ZIP 244.

The transaction digests for transaction identifiers and signatures will include the newly added parts of the transaction, but not the witness information. That is, they will not include the witness fields of TZE Input Encodings. Refer to ZIP 245 [zip-0245] for the specification of these new digests. If the changes in this ZIP are deployed, the changes that are described in ZIP 245 must be deployed also.

Rationale

A single extension type must be used in each transaction that has both TZE inputs and TZE outputs, to prevent cross-protocol attacks. The disadvantage is that, with this restriction, no TZE-encumbered value can be spent directly into a different TZE type. The value must go through a regular address in between. This restriction can be relaxed in future ZIPs for specific combinations of (type, mode) pairs that were analyzed for cross-protocol attacks. But here we select a fail-safe default behaviour.

A transaction that has TZE inputs and no TZE outputs is not subject to single-extension restrictions or single-mode restrictions. Likewise, a transaction that has TZE outputs and no TZE inputs can have TZE outputs for multiple extension-type/mode pairs. This is because the potential for cross-protocol attacks in this situation is negligible.

In an earlier draft version of this ZIP, the payloads were stored inside transparent inputs and outputs. This had the advantage that no change to the transaction format was necessary. But the consensus rules were significantly more complicated. Also, in that design, the extension logic was coupled too tightly to the transparent address logic. Instead, dedicated transaction fields and a separate unspent output set are used in this ZIP.

Security and Privacy Considerations for Future TZE Implementations

This ZIP is based on the assumption that the base transaction format is non-malleable. But the precondition and witness byte sequences are treated here as opaque. It is the responsibility of tze_verify to enforce these items:

A ZIP that contains the definition of a new extension type must include a section about how any potential sources of malleability are handled.

This ZIP includes restrictions to prevent cross-protocol attacks. But the extension mode is another potential attack surface. It is the responsibility of ZIPs that contain definitions of new extensions to include one or both of these items:

Reference Implementation

Acknowledgements

Zaki Manian suggested the handler semantics of tze_verify. These semantics are based on the design of Cosmos. Daira-Emma Hopwood and Sean Bowe gave useful feedback on an early draft of this ZIP. They also helped to analyse the various sources of transaction ID malleability.

We also thank the many other people who took part in discussions at Zcon1. These discussions led to the earlier draft version of this ZIP.

References

[BCP14]
Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"
[protocol]
Zcash Protocol Specification, Version 2021.2.16 or later
[protocol-constants]
Zcash Protocol Specification, Version 2021.2.16. Section 5.3: Constants
[protocol-txnconsensus]
Zcash Protocol Specification, Version 2021.2.16. Section 7.1: Transaction Consensus Rules
[zip-0032]
ZIP 32: Shielded Hierarchical Deterministic Wallets
[zip-0200]
ZIP 200: Network Upgrade Mechanism
[zip-0244]
ZIP 244: Transaction Non-Malleability Support
[zip-0245]
ZIP 245: Transaction Identifier Digests & Signature Validation for Transparent Zcash Extensions
[zip-draft-bolt]
Draft ZIP: Add support for Blind Off-chain Lightweight Transactions (Bolt) protocol
[librustzcash_zip222]
Rust language reference implementation of TZE API
[zcashd_zip222]
zcashd reference implementation of consensus rule changes