Zcash in Simplified Technical English

This page is the Simplified Technical English version of ZIP 202. The source ZIP is the normative text. Read the writing rules to understand the marked words.

ZIP 202: Version 3 Transaction Format for Overwinter

ZIP: 202
Title: Version 3 Transaction Format for Overwinter
Owners: Daira-Emma Hopwood <daira@jacaranda.org>
Original-Authors: Simon Liu
Status: Final
Category: Consensus
Created: 2018-01-10
License: MIT

Terminology

On this page, STE words are in the place of the BCP 14 key words of the source. Refer to the key word table.

In this document, these terms have the definitions in ZIP 200 [zip-0200]:

In this document, the term "Overwinter" has the definition in ZIP 201 [zip-0201].

Abstract

This ZIP contains the definition of a new transaction format. This format is necessary for Network Upgrade Mechanism [zip-0200] and Transaction Expiry [zip-0203].

Motivation

At its launch, Zcash had support for the version 1 transactions of upstream Bitcoin. Zcash also had a definition of a new version 2 transaction format. The version 2 format has additional fields that are necessary for shielded transactions.

VersionFieldDescriptionType
>= 1versionpositive valueint32
>= 1tx_in_countvariable-length integercompactSize
>= 1tx_inlist of inputsvector
>= 1tx_out_countvariable-length integercompactSize
>= 1tx_outlist of outputsvector
>= 1lock_timeblock height or timestampuint32
>= 2nJoinSplitvariable-length integercompactSize
>= 2vJoinSplitlist of joinsplitsvector
>= 2joinSplitPubKeyjoinsplit_sig public key32 bytes
>= 2joinSplitSigsignature64 bytes

A new transaction format is necessary for these purposes:

Specification

Transaction format version 3

There is a new version 3 transaction format for Overwinter.

The version 3 format is different from the version 2 format in these ways:

  • The header (the first four bytes, little-endian encoded):
    • fOverwintered flag: bit 31, must be set
    • nVersion: bits 30-0, positive integer
  • nVersionGroupId
  • nExpiryHeight
VersionFieldDescriptionType
>= 3headercontains:
  • fOverwintered flag (bit 31, always set)
  • nVersion (bits 30-0)
uint32
>= 3nVersionGroupIdversion group ID (not zero)uint32
>= 1tx_in_countvariable-length integercompactSize
>= 1tx_inlist of inputsvector
>= 1tx_out_countvariable-length integercompactSize
>= 1tx_outlist of outputsvector
>= 1lock_timeblock height or timestampuint32
>= 3expiryHeightblock heightuint32
>= 2nJoinSplitvariable-length integercompactSize
>= 2vJoinSplitlist of joinsplitsvector
>= 2joinSplitPubKeyjoinsplit_sig public key32 bytes
>= 2joinSplitSigsignature64 bytes

Header Field

The first four bytes of pre-Overwinter transactions and of Overwinter transactions are little-endian encoded.

Version 1 transaction (txid 5c6ba844e1ca1c8083cd53e29971bd82f1f9eea1f86c1763a22dd4ca183ae061):

  • It starts with the little-endian byte sequence [0x01, 0x00, 0x00, 0x00].
  • This sequence is deserialized as a 32-bit signed integer with a decimal value of 1.

Version 2 transaction (txid 4435bf8064e74f01262cb1725fd9b53e600fa285950163fd961bed3a64260d8b):

  • It starts with the little-endian byte sequence [0x02, 0x00, 0x00, 0x00].
  • This sequence is deserialized as a 32-bit signed integer with a decimal value of 2.

For the transaction parsers of Zcash versions before Overwinter, it is necessary that the transaction version number is positive. This is also true for the transaction parsers of most other Bitcoin forks.

In the version 3 transaction format, the first four bytes of a serialized transaction are the 32-bit header. The header has two fields, as shown in the table above:

  • A 1-bit fOverwintered flag. This flag must be set.
  • A 31-bit unsigned int for the version.

Pre-Overwinter parsers deserialize these four bytes as a 32-bit signed integer. With two's complement integers, the sign of an integer (positive or negative) depends on its most significant bit. When the Overwinter flag is set, the transaction version is negative. Thus pre-Overwinter parsers reject the transaction as invalid. The result is transaction replay protection between pre-Overwinter implementations and Overwinter implementations.

The example that follows is for a serialized version 3 transaction.

Pre-Overwinter parser:

  • The data starts with this little-endian byte sequence: [0x03, 0x00, 0x00, 0x80].
  • The parser deserializes the data as a 32-bit signed integer.
    • The hexadecimal value is 0x80000003 (the most significant bit is set).
    • The decimal value is -2147483645.

Pre-Overwinter parsers expect a positive value for the version, such as 1 or 2. Thus they reject the Overwinter transaction as invalid.

Overwinter parser:

  • The data starts with this little-endian byte sequence: [0x03, 0x00, 0x00, 0x80].
  • The parser deserializes the data as a 32-bit unsigned integer.
    • The binary value is 0b10000000000000000000000000000011.
  • The parser divides the 32 bits into two fields:
    • The fOverwintered flag (bit 31) as a boolean. The parser expects that this flag is set.
    • The version (bits 30 - bit 0) as an unsigned integer. The parser expects a decimal value of 3.

Overwinter parsers accept the transaction as valid, because the most significant bit of the header is set. The parser can mask off (unset) the most significant bit to get the transaction version number:

0x80000003 & 0x7FFFFFFF = 0x00000003 = 3

Version Group ID

The version group ID is an identifier of type uint32. This identifier is:

  • non-zero
  • random
  • unique

The version group ID is assigned to a transaction format version, or to a group of soft-forking transaction format versions. With the version group ID, it is easier for nodes to see the difference between consensus branches that have the same version number.

That is, with this ID, an implementation on one branch of the network does not try to parse transactions intended for a different consensus branch. This is applicable when the transactions have the same format version number but their specifications are different. For example, Zcash and a clone of Zcash can each define a custom v3 transaction format. But each has its own unique version group ID. Thus they can reject v3 transactions that have unknown version group IDs.

The transaction format is uniquely defined by the combination of the transaction version and the version group ID, nVersion || nVersionGroupId. Thus parsers can reject the transactions that are from outside the chain of the implementation and that they cannot parse.

For some new transaction versions, a network upgrade is necessary. By convention, the expectation is that a new unique version group ID is assigned to such a transaction version.

But some new transaction versions can be correctly parsed according to the format of a previous version. That is, such a transaction version only has changes of these types:

  • restrictions on the format
  • definitions of fields that were reserved before, and that old parsers can safely ignore

For such a transaction version, the same version group ID can be used again.

Expiry Height

The definition of the expiry height field is in the Transaction Expiry ZIP [zip-0203]. This field contains a block height. After that block height, the transaction cannot be mined.

Transaction Validation

A valid Overwinter transaction intended for Zcash must have:

  • version number 3; and
  • version group ID 0x03C48270; and
  • the fOverwintered flag set.

Overwinter validators must reject a transaction for a violation of the consensus rules if:

  • the fOverwintered flag is not set; or
  • the version group ID is unknown; or
  • the version number is unknown.

In the validation of version 3 transactions, the signature validation process from the Transaction Signature Validation for Overwinter ZIP [zip-0143] must be used.

Implementation

The comments and code samples in this section apply to the reference C++ implementation of Zcash. Other implementations can be different.

Transaction Version

The transaction version stays a positive value. On the main Zcash chain, transaction versions continue to be in ascending order, by convention.

Tests can continue to check that forwards-compatible transaction fields exist. To do this, the tests check the transaction version with comparison operators:

if (tx.nVersion >= 2) {
    for (int js = 0; js < joinsplits; js++) {
        ...
    }
}

During serialization or deserialization of v3 transactions, the version group ID must also be checked. The reason is that the transaction is possibly intended for a consensus branch that has a different format for its version 3 transaction:

if (tx.nVersion == 3 && tx.nVersionGroupId == OVERWINTER_VERSION_GROUP_ID) {
    auto expiryHeight = tx.nExpiryHeight;
}

Tests can continue to set the version to zero as an error condition:

mtx.nVersion = 0

Overwinter Validation

To test if the format of an Overwinter transaction is v3 or not:

if (tx.fOverwintered && tx.nVersion == 3) {
    // Valid v3 format transaction
}

This is only a test that the format of the transaction agrees with the v3 specification above.

To test if the format of an Overwinter transaction is v3 and if the transaction itself is intended for the chain of the implementation:

if (tx.fOverwintered &&
    tx.nVersionGroupId == OVERWINTER_VERSION_GROUP_ID) &&
    tx.nVersion == 3) {
    // Valid v3 format transaction intended for this client's chain
}

The expectation is that this test that includes nVersionGroupId is only necessary during the creation or deserialization of a transaction. For example, this is when the system receives an external transaction.

But it is possible that a clone of Zcash uses the same version group ID and passes the conditional.

In the end, an implementation can find if a transaction is truly intended for its chain or not. To do this, the implementation follows the signature validation process from the Transaction Signature Validation for Overwinter ZIP [zip-0143].

Deployment

This ZIP is deployed with the Overwinter network upgrade. The proposal for the activation height is in [zip-0201].

Backwards compatibility

With this ZIP, there is intentionally a change that is known as a "bilateral consensus rule change" [zip-0200]. This change is between pre-Overwinter implementations and Overwinter-compatible implementations. To use this new transaction format, it is necessary that all network participants upgrade their implementations to a compatible version in the upgrade window. Pre-Overwinter implementations treat Overwinter transactions as invalid.

After the activation of Overwinter, Overwinter-compatible implementations reject version 1 and version 2 transactions. After that activation, they accept a transaction only if it has a supported transaction version number and a recognized version group ID.

Reference Implementation

https://github.com/zcash/zcash/pull/2925

References

[BCP14]
Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"
[zip-0143]
ZIP 143: Transaction Signature Validation for Overwinter
[zip-0200]
ZIP 200: Network Upgrade Mechanism
[zip-0201]
ZIP 201: Network Handshaking for Overwinter
[zip-0203]
ZIP 203: Transaction Expiry