This page is the Simplified Technical English version of ZIP 202. The source ZIP is the normative text. Read the writing rules to understand the marked words.
ZIP 202: Version 3 Transaction Format for Overwinter
ZIP: 202 Title: Version 3 Transaction Format for Overwinter Owners: Daira-Emma Hopwood <daira@jacaranda.org> Original-Authors: Simon Liu Status: Final Category: Consensus Created: 2018-01-10 License: MIT
Terminology
On this page, STE words are in the place of the BCP 14 key words of the source. Refer to the key word table.
In this document, these terms have the definitions in ZIP 200 [zip-0200]:
- "consensus branch"
- "network upgrade"
- "consensus rule change"
In this document, the term "Overwinter" has the definition in ZIP 201 [zip-0201].
Abstract
This ZIP contains the definition of a new transaction format. This format is necessary for Network Upgrade Mechanism [zip-0200] and Transaction Expiry [zip-0203].
Motivation
At its launch, Zcash had support for the version 1 transactions of upstream Bitcoin. Zcash also had a definition of a new version 2 transaction format. The version 2 format has additional fields that are necessary for shielded transactions.
| Version | Field | Description | Type |
|---|---|---|---|
| >= 1 | version | positive value | int32 |
| >= 1 | tx_in_count | variable-length integer | compactSize |
| >= 1 | tx_in | list of inputs | vector |
| >= 1 | tx_out_count | variable-length integer | compactSize |
| >= 1 | tx_out | list of outputs | vector |
| >= 1 | lock_time | block height or timestamp | uint32 |
| >= 2 | nJoinSplit | variable-length integer | compactSize |
| >= 2 | vJoinSplit | list of joinsplits | vector |
| >= 2 | joinSplitPubKey | joinsplit_sig public key | 32 bytes |
| >= 2 | joinSplitSig | signature | 64 bytes |
A new transaction format is necessary for these purposes:
- To give support for safe network upgrades, as specified in Network Upgrade Mechanism [zip-0200].
- To give replay protection between pre-Overwinter and Overwinter consensus branches during network upgrades.
- To give replay protection between different consensus branches after Overwinter.
- To let a consensus branch have support for more than one transaction version format.
- To make sure that parsers parse transaction formats uniquely across parallel consensus branches.
- To give support for transaction expiry [zip-0203].
Specification
Transaction format version 3
There is a new version 3 transaction format for Overwinter.
The version 3 format is different from the version 2 format in these ways:
- The header (the first four bytes, little-endian encoded):
fOverwinteredflag: bit 31, must be setnVersion: bits 30-0, positive integer
nVersionGroupIdnExpiryHeight
| Version | Field | Description | Type |
|---|---|---|---|
| >= 3 | header | contains:
| uint32 |
| >= 3 | nVersionGroupId | version group ID (not zero) | uint32 |
| >= 1 | tx_in_count | variable-length integer | compactSize |
| >= 1 | tx_in | list of inputs | vector |
| >= 1 | tx_out_count | variable-length integer | compactSize |
| >= 1 | tx_out | list of outputs | vector |
| >= 1 | lock_time | block height or timestamp | uint32 |
| >= 3 | expiryHeight | block height | uint32 |
| >= 2 | nJoinSplit | variable-length integer | compactSize |
| >= 2 | vJoinSplit | list of joinsplits | vector |
| >= 2 | joinSplitPubKey | joinsplit_sig public key | 32 bytes |
| >= 2 | joinSplitSig | signature | 64 bytes |
Header Field
The first four bytes of pre-Overwinter transactions and of Overwinter transactions are little-endian encoded.
Version 1 transaction (txid 5c6ba844e1ca1c8083cd53e29971bd82f1f9eea1f86c1763a22dd4ca183ae061):
- It starts with the little-endian byte sequence [0x01, 0x00, 0x00, 0x00].
- This sequence is deserialized as a 32-bit signed integer with a decimal value of 1.
Version 2 transaction (txid 4435bf8064e74f01262cb1725fd9b53e600fa285950163fd961bed3a64260d8b):
- It starts with the little-endian byte sequence [0x02, 0x00, 0x00, 0x00].
- This sequence is deserialized as a 32-bit signed integer with a decimal value of 2.
For the transaction parsers of Zcash versions before Overwinter, it is necessary that the transaction version number is positive. This is also true for the transaction parsers of most other Bitcoin forks.
In the version 3 transaction format, the first four bytes of a serialized transaction are the 32-bit header. The header has two fields, as shown in the table above:
- A 1-bit
fOverwinteredflag. This flag must be set. - A 31-bit unsigned int for the version.
Pre-Overwinter parsers deserialize these four bytes as a 32-bit signed integer. With two's complement integers, the sign of an integer (positive or negative) depends on its most significant bit. When the Overwinter flag is set, the transaction version is negative. Thus pre-Overwinter parsers reject the transaction as invalid. The result is transaction replay protection between pre-Overwinter implementations and Overwinter implementations.
The example that follows is for a serialized version 3 transaction.
Pre-Overwinter parser:
- The data starts with this little-endian byte sequence: [0x03, 0x00, 0x00, 0x80].
- The parser deserializes the data as a 32-bit signed integer.
- The hexadecimal value is 0x80000003 (the most significant bit is set).
- The decimal value is -2147483645.
Pre-Overwinter parsers expect a positive value for the version, such as 1 or 2. Thus they reject the Overwinter transaction as invalid.
Overwinter parser:
- The data starts with this little-endian byte sequence: [0x03, 0x00, 0x00, 0x80].
- The parser deserializes the data as a 32-bit unsigned integer.
- The binary value is 0b10000000000000000000000000000011.
- The parser divides the 32 bits into two fields:
- The
fOverwinteredflag (bit 31) as a boolean. The parser expects that this flag is set. - The version (bits 30 - bit 0) as an unsigned integer. The parser expects a decimal value of 3.
- The
Overwinter parsers accept the transaction as valid, because the most significant bit of the header is set. The parser can mask off (unset) the most significant bit to get the transaction version number:
0x80000003 & 0x7FFFFFFF = 0x00000003 = 3
Version Group ID
The version group ID is an identifier of type uint32. This identifier is:
- non-zero
- random
- unique
The version group ID is assigned to a transaction format version, or to a group of soft-forking transaction format versions. With the version group ID, it is easier for nodes to see the difference between consensus branches that have the same version number.
That is, with this ID, an implementation on one branch of the network does not try to parse transactions intended for a different consensus branch. This is applicable when the transactions have the same format version number but their specifications are different. For example, Zcash and a clone of Zcash can each define a custom v3 transaction format. But each has its own unique version group ID. Thus they can reject v3 transactions that have unknown version group IDs.
The transaction format is uniquely defined by the combination of the transaction version and the version group ID, nVersion || nVersionGroupId. Thus parsers can reject the transactions that are from outside the chain of the implementation and that they cannot parse.
For some new transaction versions, a network upgrade is necessary. By convention, the expectation is that a new unique version group ID is assigned to such a transaction version.
But some new transaction versions can be correctly parsed according to the format of a previous version. That is, such a transaction version only has changes of these types:
- restrictions on the format
- definitions of fields that were reserved before, and that old parsers can safely ignore
For such a transaction version, the same version group ID can be used again.
Expiry Height
The definition of the expiry height field is in the Transaction Expiry ZIP [zip-0203]. This field contains a block height. After that block height, the transaction cannot be mined.
Transaction Validation
A valid Overwinter transaction intended for Zcash must have:
- version number 3; and
- version group ID 0x03C48270; and
- the
fOverwinteredflag set.
Overwinter validators must reject a transaction for a violation of the consensus rules if:
- the
fOverwinteredflag is not set; or - the version group ID is unknown; or
- the version number is unknown.
In the validation of version 3 transactions, the signature validation process from the Transaction Signature Validation for Overwinter ZIP [zip-0143] must be used.
Implementation
The comments and code samples in this section apply to the reference C++ implementation of Zcash. Other implementations can be different.
Transaction Version
The transaction version stays a positive value. On the main Zcash chain, transaction versions continue to be in ascending order, by convention.
Tests can continue to check that forwards-compatible transaction fields exist. To do this, the tests check the transaction version with comparison operators:
if (tx.nVersion >= 2) {
for (int js = 0; js < joinsplits; js++) {
...
}
}
During serialization or deserialization of v3 transactions, the version group ID must also be checked. The reason is that the transaction is possibly intended for a consensus branch that has a different format for its version 3 transaction:
if (tx.nVersion == 3 && tx.nVersionGroupId == OVERWINTER_VERSION_GROUP_ID) {
auto expiryHeight = tx.nExpiryHeight;
}
Tests can continue to set the version to zero as an error condition:
mtx.nVersion = 0
Overwinter Validation
To test if the format of an Overwinter transaction is v3 or not:
if (tx.fOverwintered && tx.nVersion == 3) {
// Valid v3 format transaction
}
This is only a test that the format of the transaction agrees with the v3 specification above.
To test if the format of an Overwinter transaction is v3 and if the transaction itself is intended for the chain of the implementation:
if (tx.fOverwintered &&
tx.nVersionGroupId == OVERWINTER_VERSION_GROUP_ID) &&
tx.nVersion == 3) {
// Valid v3 format transaction intended for this client's chain
}
The expectation is that this test that includes nVersionGroupId is only necessary during the creation or deserialization of a transaction. For example, this is when the system receives an external transaction.
But it is possible that a clone of Zcash uses the same version group ID and passes the conditional.
In the end, an implementation can find if a transaction is truly intended for its chain or not. To do this, the implementation follows the signature validation process from the Transaction Signature Validation for Overwinter ZIP [zip-0143].
Deployment
This ZIP is deployed with the Overwinter network upgrade. The proposal for the activation height is in [zip-0201].
Backwards compatibility
With this ZIP, there is intentionally a change that is known as a "bilateral consensus rule change" [zip-0200]. This change is between pre-Overwinter implementations and Overwinter-compatible implementations. To use this new transaction format, it is necessary that all network participants upgrade their implementations to a compatible version in the upgrade window. Pre-Overwinter implementations treat Overwinter transactions as invalid.
After the activation of Overwinter, Overwinter-compatible implementations reject version 1 and version 2 transactions. After that activation, they accept a transaction only if it has a supported transaction version number and a recognized version group ID.
Reference Implementation
References
- [BCP14]
- Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words"
- [zip-0143]
- ZIP 143: Transaction Signature Validation for Overwinter
- [zip-0200]
- ZIP 200: Network Upgrade Mechanism
- [zip-0201]
- ZIP 201: Network Handshaking for Overwinter
- [zip-0203]
- ZIP 203: Transaction Expiry