Skip to content

Latest commit

 

History

History
632 lines (519 loc) · 79.7 KB

README.md

File metadata and controls

632 lines (519 loc) · 79.7 KB

Zcashd

Important Information

ZIP-400 documents the schema used for zcashd v3.0.0-rc1. Since then, the format has changed a bit. This issue also tracks ZIP-400's update.

Background

The following text is taken from this wallet guide for exchanges. Read through to learn more about changes accross versions.

The zcashd internal wallet was, as with the rest of zcashd, forked from Bitcoin code in May 2015. The original design of the wallet treated all transparent addresses in the wallet as receiving funds into a single "bucket of money" from which spends could be made. In this original design, individual addresses were not treated as having individual, distinguishable balances (even though such per-address balances were representable at the protocol layer). This single "bucket" of transparent funds was treated as though it was associated with a single spending authority, even though actually spending funds from this pool might involve creating signatures with multiple distinct transparent spending keys. The RPC method getnewaddress produced new keys internally via derivation from system randomness, and so these keys had to be backed up independently even though the wallet did not make distinctions between them visible to the user of zcashd. Similarly, the getbalance RPC method treated funds spendable by these independent keys uniformly, as did the other Bitcoin-inherited methods.

When zcashd introduced the Sprout, and later the Sapling protocols, it diverged from this original design by treating each Sprout and Sapling address in the wallet as being associated with an independent spending authority tied to the address. With Sprout, keys continued to be derived from system randomness, but Sapling introduced a new hierarchical derivation mechanism, similar to that defined in BIPs 32, 43 and 44. Instead of deriving keys from randomness, Sapling keys were all derived from a single binary seed value.

As part of introducing the Sprout transfer protocol zcashd introduced a few new RPC methods, most importantly z_getbalance and z_sendmany which reflected the choice to treat separate addresses as holding independent balances, and these semantics were retained when the Sapling protocol was added. However, in the process, a conceptual error was introduced.

With the introduction of z_getbalance and z_sendmany, it became possible for users to begin treating separate transparent addresses in the wallet as having independent balances, even though the Bitcoin-derived RPC methods treated those balances as simply being part of a larger undifferentiated pool of funds. Over the intervening years, users have come to depend upon this inadvertent semantic change.

Key derivation

zcashd version 4.6.0 and later uses this path to derive "legacy" Sapling addresses from a mnemonic seed phrase under account 0x7FFFFFFF, using hardened derivation for address_index:

m/purpose'/coin_type'/account'/address_index

The following text is taken from this Github issue.

When Sapling was released, zcashd implemented HD derivation of Sapling addresses in a fashion that was inconsistent with HD derivation according to BIP 44. In version 4.7.0 zcashd introduced HD derivation from a mnemonic seed according to BIP 32 and BIP 44, with a nonstandard accommodation in the generation of the mnemonic seed to make it possible to also reproduce previously derived Sapling keys. This accommodation needs to be documented, along with the process for correct discovery of such previously-derived Sapling keys.

In addition, in order to continue allow zcashd's legacy transparent APIs such as getnewaddress and z_getnewaddress to continue to function, zcashd introduced the idea of the ZCASH_LEGACY_ACCOUNT constant for use in address derivation consistent with the previous semantics of those methods. Derivation of keys under ZCASH_LEGACY_ACCOUNT is also nonstandard with respect to BIP 32 and BIP 44, and so needs to be properly documented here in order to make it possible for other wallet implementations to correctly rediscover funds controlled by keys derived using this mechanism.

Note that the primary derivation path is defined in ZIP-32.

Serialization framework overview

Bitcoind (and zcashd, by extension) employs a custom serialization framework to encode both Key and Value fields. This framework handles type-specific serialization, compact sizes, optional fields, and more. See the serialization reference for details on how each type is serialized. Also check out the Bitcoin Core Onboarding's wallet database section.

Data is stored in dat files, which are implemented as a BerkeleyDB1. Each entry is serialized using the following structure:

<keyname_length><keyname><key>
<value>

Where:

  • <keyname_length> is a byte representing the length of <keyname>.
  • <keyname> is an ASCII encoded string of the length <keyname_length> and <key> the binary data.
  • <key> is the output of the serialization of each Key.
  • <value> is the output of the serialization of each Value.

Each key and value has an associated C++ class (and sometimes a Rust struct, depending on the version) from zcashd. Check this table to learn more about how each class is serialized.

Serialization of Key/Value pairs by version

What value/pair is serialized is taken from this walletdb.cpp file.

The following key-values can be found: the property names in bold mean only one instance of this type can exist in the entire database, while the others, suffixed by '*' can have multiple instances. key and value columns of the table contain the types that the stored data is representing.

v3.0.0-rc1

Taken from: https://zips.z.cash/zip-0400. Open in fullscreen, as this table is too wide.

keyname key value Description
acc* string (account name) CAccount DISABLED. Account information.
acentry* string (account name) + uint64_t (counter) CAccountingEntry DISABLED. Account entry. Tracks internal transfers.
bestblock - CBlockLocator The current best block of the blockchain.
chdseed uint256 (seed fingerprint) vector<unsigned char> (see Encryption) Encrypted HD seed.
ckey* CPubKey vector<unsigned char> (see Encryption) Transparent pubkey and encrypted private key.
csapzkey* libzcash::SaplingIncomingViewingKey libzcash::SaplingExtendedFullViewingKey (extended full viewing key) + vector<unsigned char> (crypted secret, see Encryption) Sapling pubkey and encrypted private key.
cscript uint160 CScriptBase Serialized script, used inside transaction inputs and outputs.
czkey* libzcash::SproutPaymentAddress uint256 + vector<unsigned char> Encrypted Sprout pubkey and private key.
defaultkey - CPubKey Default Transparent key.
destdata* string (address) + string (key) string (value) Adds destination data tuple to the store.
hdchain - CHDChainV3 Hierarchical Deterministic chain code, derived from seed.
hdseed* uint256 (BLAKE2b hash) RawHDSeed Hierarchical Deterministic seed.2
key* CPubKey CPrivKey + HASH256(CPubKey + CPrivKey) Transparent pubkey and privkey.
keymeta* CPubKey CKeyMetadata Transparent key metadata.
minversion - int (check wallet versions) Wallet required minimal version.
mkey unsigned int CMasterKey Master key, used to encrypt public and private keys of the db.
name* string (address) string (name) Name of an address to insert in the address book.
orderposnext - int64_t Index of next tx.
pool* int64_t CKeyPool An address look-ahead pool.3
purpose* string (address) string (purpose) Short description or identifier of an address.
sapzaddr* libzcash::SaplingPaymentAddress libzcash::SaplingIncomingViewingKey Sapling z-addr Incoming Viewing key and address.
sapextfvk* libzcash::SaplingExtendedFullViewingKey char = '1' Sapling Extended Full Viewing Key.
sapzkey* libzcash::SaplingIncomingViewingKey libzcash::SaplingExtendedSpendingKey Sapling Incoming Viewing Key and Extended Spending Key.
tx* uint256 (hash) CWalletTx Store all transactions that are related to wallet.
version - int (check wallet versions) The CLIENT_VERSION from clientversion.h.
vkey* libzcash::SproutViewingKey char = '1' Sprout Viewing Keys.
watchs* CScriptBase char = '1' Watch-only t-addresses.
witnesscachesize - int64_t (witness cache size) Shielded Note Witness cache size.
wkey* - - Wallet key. No longer used
zkey* libzcash::SproutPaymentAddress libzcash::SproutSpendingKey Sprout Payment Address and Spending Key.
zkeymeta* libzcash::SproutPaymentAddress CKeyMetadata Sprout Payment Address and key metadata.

Notes

The 'Account' API, which included both acc and acentry and were inherited from Bitcoin Core, has been disabled since the launch of Zcash. and finally removed in zcashd v4.5.0. Read the zcashd v4.5.0 release notes and this bitcoin-core v0.17.0 release notes for more information.

v4.0.0

No changes to storage format since v3.0.0

Check out the full diff here

v4.5.0

Removed Fields:

keyname key value Description
acc string CAccount -
acentry string + uint64_t CAccountingEntry -

v5.0.0

Added and Removed Fields:

keyname key value Description
hdseed uin256 RawHDSeed -
chdseed uin256 vector<unsigned char> -
networkinfo - pair<string = 'Zcash', string (network identifier)> Network identifier.
orchard_note_commitment_tree - OrchardWalletNoteCommitmentTreeWriter Orchard note commitment tree.
unifiedaccount ZcashdUnifiedAccountMetadata 0x00 Unified account information.
unifiedfvk libzcash::UFVKId libzcash::UnifiedFullViewingKey::Encode(string, UnifiedFullViewingKeyPtr) as string Encoded unified FVK.
unifiedaddrmeta ZcashdUnifiedAddressMetadata 0x00 Unified address metadata.
mnemonicphrase uint256 (seed fingerprint) MnemonicSeed (seed) Mnemonic phrase.
cmnemonicphrase uint256 std::vector<unsigned char> (encrypted mnemonic seed. Check Encryption for more information) Encrypted mnemonic phrase.
mnemonichdchain - CHDChainV5 HD chain metadata.
recipientmapping* pair<uint256, CSerializeRecipientAddress> string (recipient) Maps transaction to recipient UA.

Check out the full diff here

v6.0.0

Added Fields:

keyname key value Description
bestblock - CBlockLocator (empty) The current best block of the blockchain. Empty block locator so versions that require a merkle branch automatically rescan.
bestblock_nomerkle - CBlockLocator A place in the block chain. If another node doesn't have the same branch, it can find a recent common trunk.

Check out the full diff here

Serialization reference

Common data types

Note on signed integer serialization: All signed integers are serialized using two's complement representation. This format is standard for representing signed numbers in binary and is compatible with the little-endian encoding used throughout the bitcoind/zcashd serialization framework.

Data Type Description Serialized as
int 32-bit signed integer. Little-endian, 4 bytes
unsigned int 32-bit unsigned integer. Little-endian, 4 bytes
int32_t 32-bit signed integer. Little-endian, 4 bytes
int64_t 64-bit signed integer. Little-endian, 8 bytes
uint8_t 8-bit unsigned integer. 1 byte
uint32_t 32-bit unsigned integer. Little-endian, 4 bytes
uint64_t 64-bit unsigned integer. Little-endian, 8 bytes
uint160 An opaque blob of 160 bits without integer operations. Little-endian, 20 bytes
uint256 An opaque blob of 256 bits without integer operations. Little-endian, 32 bytes
uint252 Wrapper of uint256 with guarantee that first four bits are zero. Little-endian, 32 bytes
string UTF-8 encoded string. 1 byte (length) + bytes of the string
unsigned char Byte or octet. 1 byte
bool Boolean value. 1 byte (0x00 = false, 0x01 = true)
pair<K, T> A pair of 2 elements of types K and T. T and K in sequential order.
CCompactSize A variable-length encoding for collection sizes. 1 byte for sizes < 253, 3 bytes for sizes between 253 and 65535, 5 bytes for sizes between 65536 and 4GB, 9 bytes for larger sizes.
array<T, N> An array of N elements of type T. Serialized elements T in order. N is not serialized, as the array is always the same length.
vector<T> Dynamic array of elements of type T. CCompactSize (number of elements) + serialized elements T in order.
prevector<N, T> Dynamic array of elements of type T, optimized for fixed size. CCompactSize (number of elements) + serialized elements T in order.
map<K, V> A map of key-value pairs. CCompactSize (number of key-value pairs) + serialized keys K and values V in order.
optional<T> A container that optionally holds a value, serialized with a presence flag followed by the value if present. 1 byte (discriminant: 0x00 = absent, 0x01 = present) + serialized value T if present.
list<T> Dynamically sized linked list of elements of type T. CCompactSize (number of elements) + serialized elements T in order.
libzcash::diversifier_t An 11-byte value used to select a valid Jubjub base point, which is then used to derive a diversified Sapling or Orchard address. array<unsigned char>[11]
libzcash::diversifier_index_t An opaque blob of 88 bits, representing a diversifier index. array<unsigned char>[11]
CAmount A wrapper for int64_t that represents monetary values in zatoshis. int64_t
joinsplit_sig_t The JoinSplit signature, an Ed25519 digital signature. array<unsigned char>[64]
binding_sig_t A Sapling binding signature (a RedJubjub signature) that enforces consistency between Spend descriptions and Output descriptions. array<unsigned char>[64]
mapSproutNoteData_t Mapping of (Sprout) note outpoints to note data. map<JSOutPoint,SproutNoteData>
mapSaplingNoteData_t Mapping of (Sapling) note outpoints to note data. map<SaplingOutPoint,SaplingNoteData>
spend_auth_sig_t Signature authorizing a spend. array<unsigned char>[64]

Classes

Class Description Serialized as
CAccount Account information. CPubKey (public key)
CPubKey Public key. CCompactSize (public key length) + unsigned char[33 | 65](public key in compressed/uncompressed format)
CPrivKey Uncompressed private key, encoded as a DER ECPrivateKey type from section C.4 of SEC 1.4 5 vector<unsigned char>[214 | 279] (private key)
CKeyMetadata Key metadata. int64_t (creation time as unix timestamp. 0 if unknown) + string (optional HD/zip32 keypath2) + uint256 (seed fingerprint)
CMasterKey Master key for wallet encryption. Encrypted using AES-256-CBC.6 vector<unsigned char>[32] (encryption key) + unsigned char[8] (salt) + unsigned int (0 = EVP_sha5127 | 1 = scrypt8) + unsigned int (derivation iterations) + vector<unsigned char> (extra parameters)
CAccountingEntry Tracks an internal account transfer. int64_t (credit or debit in zatoshis) + int64_t (unix timestamp) + string (other_account) + '\0' + map<string, string> (metadata, includes n to indicate position) + map<string, string> (extra information)
CBlockLocator A list of current best blocks. vector<uint256> (vector of block hashes)
CKeyPool Pre-generated public key for receiving funds or change. int64_t (creation time as unix timestamp) + CPubKey (public key)
CHDChainV3 HD chain metadata. int (nVersion) + uint256 (seed fingerprint) + int64_t (nTime) + uint32_t (accountCounter)
CHDChainV5 HD chain metadata. int (nVersion = '1') + uint256 (seed fingerprint) + int64_t (nCreateTime) + uint32_t (accountCounter) + uint32_t (legacyTKeyExternalCounter) + uint32_t (legacyTKeyInternalCounter) + uint32_t (legacySaplingKeyCounter) + bool (mnemonicSeedBackupConfirmed)
RawHDSeed Hierarchical Deterministic seed.2 vector<unsigned char>[32] (raw HD seed, min length 32)
COutPoint A combination of a transaction hash and an index n into its vout. uint256 (hash) + uint32_t (index)
SaplingOutPoint A combination of a transaction hash and an index n into its sapling output description (vShieldedOutput). uint256 (hash) + uint32_t (index)
CTxIn An input of a transaction. COutPoint (previous tx output) + CScriptBase (script signature) + uint32_t (sequence number)
CTxOut An output of a transaction. Contains the public key that the next input must sign to claim it. int64_t (value) + CScriptBase (scriptPubKey)
JSOutPoint JoinSplit note outpoint. uint256 (hash) + uint64_t (index into CTransaction.vJoinSplit) + uint8_t (index into JSDescription fields)
SproutNoteData Data for a Sprout note. libzcash::SproutPaymentAddress (address) + optional<uint256> (nullifier) + list<SproutWitness> (witnesses) + int (witnessHeight)
IncrementalMerkleTree<Depth, Hash> Incremental Merkle tree with Depth levels, using Hash as the hash function. optional<Hash> (left) + optional<Hash> (right) + vector<optional<Hash>> (parents)
IncrementalWitness<Depth, Hash> Incremental Merkle witness. IncrementalMerkleTree<Depth, Hash> (tree) + vector<Hash> (filled/hashed nodes) + optional<IncrementalMerkleTree<Depth, Hash>> (cursor)
libzcash::PedersenHash Pedersen hash. uint256 (hash)
SaplingWitness An incremental witness that tracks the inclusion of a note commitment in the Sapling Merkle tree. IncrementalWitness<32, libzcash::PedersenHash>
SaplingNoteData Data for a Sapling note. libzcash::SaplingIncomingViewingKey (ivk) + optional<uint256> (nullifier) + list<SaplingWitness> (witnesses) + int (witnessHeight)
CompressedG1 Compressed point in G1. unsigned char (0x02 or 0x02 | 1) + uint256 (x)
CompressedG2 Compressed point in G2. unsigned char (0x0a or 0x0a | 1) + uint256 (x)
libzcash::GrothProof Groth proof. array<unsigned char>[192] (48 (π_A) + 96 (π_B) + 48 (π_C))
libzcash::PHGRProof Compressed zkSNARK proof.9 CompressedG1 (g_A) + CompressedG1 (g_A_prime) + CompressedG2 (g_B) + CompressedG1 (g_B_prime) + CompressedG1 (g_C) + CompressedG1 (g_C_prime) + CompressedG1 (g_K) + CompressedG1 (g_H)
SpendDescription Describes a Spend transfer. uint256 (commitment value) + uint256 (anchor value) + uint256 (nullifier) + uint256 (rk) + libzcash::GrothProof (zkproof) + spend_auth_sig_t (spendAuthSig)
libzcash::SaplingEncCiphertext Ciphertext for the recipient to decrypt. array<unsigned char>[580]((1 + 11 + 8 + 32 + 512) + 16)
libzcash::SaplingOutCiphertext Ciphertext for outgoing viewing key to decrypt. array<unsigned char>[80] ((32 + 32) + 16)
OutputDescription Shielded output to a transaction. Contains data that describes an Output transfer. uint256 (commitment value) + uint256 (cmu) + uint256 (ephemeralKey) + libzcash::SaplingEncCiphertext (encCiphertext) + libzcash::SaplingOutCiphertext (outCiphertext) + libzcash::GrothProof (zkproof)
JSDescription JoinSplit description. CAmount (vpub_old) + CAmount (vpub_new) + uint256 (anchor) + array<uint256>[2] (nullifiers) + array<uint256>[2] (commitments) + uint256 (ephemeralKey) + uint256 (randomSeed) + array<uint256>[2] (message auth codes)
OrchardWalletTxMeta A container for storing information derived from a tx that is relevant to restoring Orchard wallet caches. map<uint32_t, libzcash::OrchardIncomingViewingKey> (mapOrchardActionData) + vector<uint32_t> (vActionsSpendingMyNotes)
SaplingBundle The Sapling component of an authorized v5 transaction. Check SaplingBundle
OrchardBundle The Orchard component of an authorized transaction. Check OrchardBundle
SaplingV4Writer Writer for the Sapling components of a v4 transaction (Sapling bundle), excluding binding signature. int64_t (zat balance. Sapling spends - Sapling outputs) + vector<SpendDescription> (shielded spends) + vector<OutputDescription> (shielded outputs)
ed25519::VerificationKey Ed25519 public key. array<uint8_t>[32]
ed25519::Signature Ed25519 digital signature. array<uint8_t>[64]
CTransaction The basic transaction that is broadcasted on the network and contained in blocks. Check CTransaction
CMerkleTx A transaction with a merkle branch linking it to the block chain. CTransaction + uint256 (hashBlock) + vector<uint256> (vMerkleBranch) + int (nIndex)
CWalletTx A transaction with additional information. Check CWalletTx
libzcash::OrchardIncomingViewingKey Orchard incoming viewing key. array<unsigned char>[32] (diversifier key) + array<unsigned char>[32] (ivk nonzero pallas scalar representation)
libzcash::SaplingFullViewingKey Sapling full viewing key. uint256 (ak) + uint256 (nk) + uint256 (ovk)
libzcash::SaplingExpandedSpendingKey Sapling expanded spending key. uint256 (ask) + uint256 (nsk) + uint256 (ovk)
libzcash::SaplingIncomingViewingKey A 32-byte value representing the incoming viewing key for a Sapling address. uint256 (32-byte ivk in little-endian, padded with zeros in the most significant bits)
libzcash::SaplingExtendedFullViewingKey Sapling extended full viewing key. uint8_t (depth) + uint32_t (parentFVKTag) + uint32_t (childIndex) + uint256 (chaincode) + libzcash::SaplingFullViewingKey (fvk) + uint256 (diversifier key)
libzcash::SaplingExtendedSpendingKey Sapling extended spending key. uint8_t (depth) + uint32_t (parentFVKTag) + uint32_t (childIndex) + uint256 (chaincode) + libzcash::SaplingExpandedSpendingKey (expsk) + uint256 (diversifier key)
libzcash::SaplingPaymentAddress Sapling payment address. diversifier_t (diversifier) + uint256 (pk_d)
libzcash::ReceivingKey Receiving key for shielded transactions. uint256 (sk_enc)
libzcash::SproutPaymentAddress Sprout payment address. uint256 (a_pk) + uint256 (pk_enc)
OrchardRawAddress Raw Orchard address. This type doesn't exist per se, but is used to avoid inline definitions. diversifier_t (diversifier) + uint256 (pk_d)
libzcash::SproutViewingKey Sprout viewing key. uint256 (a_pk) + libzcash::ReceivingKey (sk_enc)
libzcash::SproutSpendingKey Sprout spending key. uint252 (a_sk)
CScriptBase Serialized script, used inside transaction inputs and outputs. prevector<28, unsigned char> (script)
libzcash::SeedFingerprint 256-bit seed fingerprint. uint256 (seed fingerprint)
libzcash::AccountId Account identifier for HD address derivation. uint32_t (accountId)
libzcash::UFVKId An internal identifier for a unified full viewing key, derived as a blake2b hash of the serialized form of the UFVK. uint256 (ufvkId)
libzcash::ReceiverType Receiver type (P2PKH, P2SH, Sapling or Orchard). uint32_t (receiverType, 0 = P2PKH, 1 = P2SH, 2 = Sapling, 3 = Orchard)
ZcashdUnifiedAccountMetadata Metadata for a unified account. libzcash::SeedFingerprint (seed fingerprint) + uint32_t (bip44CoinType) + libzcash::AccountId (accountId) + libzcash::UFVKId (ufvkId)
ZcashdUnifiedAddressMetadata Metadata for a unified address. libzcash::UFVKId (ufvkId) + libzcash::diversifier_index_t (diversifierIndex) + vector<libzcash::ReceiverType> (serReceiverTypes)
Address Address or location of a node of the Merkle tree. unsigned char(level in merkle tree) +uint64_t (address index)
MerkleBridge<H> Information required to "update" witnesses from one state of a Merkle tree to another. Check MerkleBridge Serialization
BridgeTree Sparse representation of a Merkle tree. Check BridgeTree Serialization
OrchardWalletNoteCommitmentTreeWriter Note commitment tree for an Orchard wallet. Check OrchardWalletNoteCommitmentTreeWriter Serialization
libzcash::UnifiedFullViewingKey::Encode(string, UnifiedFullViewingKeyPtr) Serialized Ufvk. string (Bech32m-encoded network HRP combined with the jumbled and Base32-encoded representation of the HRP.10)
MnemonicSeed Mnemonic seed. uint32_t (language, more information here) + string (mnemonic)
ReceiverTypeSer Serialization wrapper for reading and writing ReceiverType in CompactSize format. CCompactSize (size) + uint64_t (receiver type: 0 = P2PKH, 1 = P2SH, 2 = Sapling, 3 = Orchard)
CSerializeRecipientAddress Recipient address.

More details

This section will serve as a reference for the classes whose serialization involves complex paths. The idea is to show conditionals in a way that are easy to read.

SaplingBundle

Taken from the write_v5_bundle function under depends/<arch>/vendored-sources/zcash_primitives/src/transaction/components/sapling.rs.

// Shielded spends (without witness data)
uint256 // (commitment value)
uint256 // (nullifier)
uint256 // (rk)

// Shielded outputs (without proof)
uint256 // (commitment value)
uint256 //(cmu)
uint256 // (ephemeral key)
libzcash::SaplingEncCiphertext // (enc_ciphertext)
libzcash::SaplingOutCiphertext // (out_ciphertexts)

if (shielded_spends.length > 0 AND shielded_outputs.length > 0) {
    int64_t // (value balance)
}

if (shielded_spends.length > 0) {
    uint256 // (shielded_spends[0].anchor; the root of the Sapling commitment tree that the first spend commits to)
}

array<
    array<uint8_t>[192] // (Groth proof bytes)
> // (shielded spends zkProofs)

array<
    array<unsigned char>[64] // (redjubjub::Signature)
> // (spends auth sigs)

array<
    array<uint8_t>[192] // (Groth proof bytes)
> //  (shielded outputs zkProofs)

array<unsigned char>[64] // (binding signatures)

OrchardBundle

Taken from the write_v5_bundle function under depends/<arch>/vendored-sources/zcash_primitives/src/transaction/components/orchard.rs.

vector<
    uint256 // (commitment to the net value created or consumed by the action)
    uint256 // (nullifier)
    uint256 // (rk)
    uint256 // (cmx, commitment to the new note being created)

    {
        uint256 // (ephemeral key)
        array<uint8_t>[580] // (encrypted note ciphertext)
        array<uint8_t>[80] // (encrypted value that allows the holder of the outgoing cipher key for the note to recover the note plaintext)
    } // (note ciphertext)
> // (actions without auth)

byte // (flags. https://zips.z.cash/protocol/protocol.pdf#txnencoding)
int64_t // (value balance, net value moved into or out of the Orchard shielded pool)
uint256 // (anchor, the root of the Orchard commitment tree that this bundle commits to)
vector<uint8_t> // (proof components of the authorizing data)
array<
    uint8_t // (authorization for an Orchard action)
>[64] // (authorizations for each orchard action)
array<uint8_t>[64] // (binding signature)

CTransaction

Taken from the SerializationOp function under src/primitives/transaction.h.

uint32_t // (header)

if (fOverwintered) {
    uint32_t // (version group id)
}
if (isZip225V5) {
    uint32_t // (consensus branch id)
    uint32_t // (nLockTime)
    uint32_t // (nExpiryHeight)

    // Transparent Transaction Fields
    vector<CTxIn> // (vin)
    vector<CTxOut> // (vout)

    // Sapling Transaction Fields
    SaplingBundle // (saplingBundle)

    // Orchard Transaction Fields
    OrchardBundle // (orchardBundle)
} else {

    // Legacy transaction formats
    vector<CTxIn> // (vin)
    vector<CTxOut> // (vout)
    uint32_t // (nLockTime)
    if (isOverwinterV3 OR isSaplingV4 OR isFuture) {
        uint_32_t // (nExpiryHeight)
    }

    SaplingV4Writer // (saplingBundle)

    if (nVersion >= 2) {
        // These fields do not depend on fOverwintered
        vector<JSDescription>> // (vJoinSplit)
        if (vJoinSplitSize > 0) {
            ed25519::VerificationKey // (joinSplitPubKey)
            ed25519::Signature // (joinSplitSig)
        }
    }
    if ((isSaplingV4 OR isFuture) AND saplingBundle.IsPresent()) {
        array<unsigned char>[64] // (sapling bundle binding signature)
    }
}

CWalletTx

Taken from the SerializationOp function under src/wallet/wallet.h.

CMerkleTx // (transaction data with a merkle branch linking it to the block chain)
vector<CMerkleTx> // (vUnused, empty. Used to be vtxPrev)
mapValue_t // (mapValue)
mapSproutNoteData_t // (mapSproutNoteData)
vector<pair<string, string>> // (vOrderForm)
unsigned int // (fTimeReceivedIsTxTime)
unsigned int // (nTimeReceived, time received by this node)
char // (fFromMe)
char = '0' // (fSpent)
if (fOverwintered AND nVersion >= 4) {
    mapSaplingNoteData_t // (mapSaplingNoteData)
}
if (fOverwintered AND nVersion >= 5) {
    OrchardWalletTxMeta // (information relevant to restoring Orchard wallet caches)
}

MerkleBridge<H: HashSer + Ord>

Taken from the write_bridge function under src/rust/src/incremental_merkle_tree.rs.

unsigned char = 2 // (serialization version, SER_V2)
optional<uint64_t> // (prior position)
vector<Address> // (node locations from ommers)
vector<Address + H (value)> // (ommers)
uint64_t (frontier position)

if (frontier.is_right_child()) {
    H // (hash)
    optional<H> // (most recent leaf)
    vector<H> // (the remaining leaves)
} else {
    H // (most recent leaf)
    optional<H> // (empty)
    vector<H> // (all leaves)
}

BridgeTree<H, u32, DEPTH>

Taken from the write_tree function under src/rust/src/incremental_merkle_tree.rs.

unsigned char = '3' // (serialization version, SER_V3)
vector<MerkleBridge> // (prior bridges)
optional<MerkleBridge> // (current bridge at the tip of this tree)
vector<
    uint64_t // (position)
    uint64_t // (index in the bridges vector)
> // (map from leaf positions that have been marked to the index of the bridge whose tip is at that position in this tree's list of bridges)
vector<
    uint32_t // (checkpoint id, block height)
    uint64_t // (prior bridges length)
    vector<uint64_t> // (the set of the positions that have been marked during the period that this checkpoint is the current checkpoint)
    vector<uint64_t> // (mark positions forgotten due to notes at those positions having been spent since the position at which this checkpoint was created)
> // (checkpoints, referring to the checkpoints to which this tree may be rewound)

OrchardWalletNoteCommitmentTreeWriter

Taken from the orchard_wallet_write_note_commitment_tree function under src/rust/src/wallet.rs.

unsigned char = '1' // (note state version, NOTE_STATE_V1)
optional<uint32_t> // (last checkpoint, block height)
BridgeTree<H, u32, DEPTH> // (commitment tree)

/// Note positions
vector<
    uint256 // (txid)
    uint256 // (tx height)
    // Action positions
    vector<
        uint32_t // (action index)
        uint64_t // (position)
    >
>

CSerializeRecipientAddresslibzcash::ReceiverType

ReceiverTypeSer // (receiver type)
if (receiverType == P2PKH) {
    uint160 // (P2PKH address)
} else if (receiverType == P2SH) {
    uint160 // (P2SH address)
} else if (receiverType == SaplingPaymentAddress) {
    libzcash::SaplingPaymentAddress // (Sapling payment address)
} else if (receiverType == OrcharRawAddress) {
    OrchardRawAddress // (Orchard raw address)
}

libzcash::UnifiedFullViewingKey::Encode(string, UnifiedFullViewingKeyPtr)

Taken from the unified_full_viewing_key_serialize function under src/rust/src/unified_keys_ffi.rs.

/*
* Bech32m encoding of (
*   HRP,
*   Jumbled padded raw encoding of the HRP, in base32
* )
*
* where HRP is the string representation of the network (main, test, regtest)
* For more information, read https://zips.z.cash/zip-0316#jumbling
*/
Bech32m(
    string, // (HRP, string representation of the network (main, test, regtest))
    f4jumble(
        {
            vector<
                uint32_t // (typecode, P2pkh, P2sh, Sapling, Orchard)
                vector<
                    /*
                    * Orchard([u8; 96]) => `(ak, nk, rivk)` each 32 bytes
                    * Sapling([u8; 128]) => `(ak, nk, ovk, dk)` each 32 bytes
                    *
                    * Pruned version of the extended public key for the BIP 44 account corresponding
                    * to the transparent address subtree from which transparent addresses are derived.
                    * This includes just the chain code (32 bytes) and the compressed public key (33 bytes).
                    * P2pkh([u8; 65]) => `(chaincode, pk)`
                    */
                    FVK
                >
            > // (raw encoding)
            array<uint8_t>[16] // (padding, HRP as bytes)
        }
    )
) // as string

Encryption

Private key encryption, along with other sensitive data, is done based on a CMasterKey, which holds a salt and random encryption key.

CMasterKeys are encrypted using AES-256-CBC6 using a key derived using derivation method nDerivationMethod (0 == EVP_sha512()) and derivation iterations nDeriveIterations. vchOtherDerivationParameters is provided for alternative algorithms which may require more parameters (such as scrypt).

Wallet Private Keys are then encrypted using AES-256-CBC with the double-sha256 of the public key as the IV, and the master key's key as the encryption key.

The CWallet class holds many CMasterKeys, which are stored in a map keyed by an unsigned integer acting as an index. Each CMasterKey is then encrypted with the user's passphrase in a function that mimics the behaviour of openssl's EVP_BytesToKey with an aes-256-cbc cipher and sha512 message digest. Because sha512's output size (64b) is greater than the aes256 block size (16b) + aes256 key size (32b), there's no need to process more than once (D_0). In other words, the passphrase is derived using the derivation function specified in the CMasterKey object, along with the other parameters. This key is then used to encrypt the key used for encrypting private keys.

When the user changes their passphrase, they are only changing the encryption applied to the CMasterKey, the inner vchCryptedKey (used to encrypt/decrypt private keys) is not changed. This means that we do not have to read all items in the wallet database, decrypt them with the old key, encrypt them with the new key, and then write them back to the database again. Instead, we only have to change the encryption applied to the CMasterKey, which is much less error-prone, and more secure.

Wallet versions

The following specifies the client version numbers for particular wallet features:

enum WalletFeature
{
    FEATURE_BASE = 10500, // the earliest version new wallets supports (only useful for getinfo's clientversion output)

    FEATURE_WALLETCRYPT = 40000, // wallet encryption
    FEATURE_COMPRPUBKEY = 60000, // compressed public keys

    FEATURE_LATEST = 60000
};

Languages

// These must match src/rust/src/zip339_ffi.rs.
// (They happen to also match the integer values correspond in the bip0039-rs crate with the
// "all-languages" feature enabled, but that's not required.)
enum Language
{
    English = 0,
    SimplifiedChinese = 1,
    TraditionalChinese = 2,
    Czech = 3,
    French = 4,
    Italian = 5,
    Japanese = 6,
    Korean = 7,
    Portuguese = 8,
    Spanish = 9,
    SIZE_HACK = 0xFFFFFFFF // needed when compiling as C
};

Example wallets

The wallet.dat files under dat_files/ (0 to 7) were generated while running the qa/zcash/full_test_suite.py tests from Zcashd.

Footnotes

  1. BerkeleyDB

  2. ZIP-32: Shielded Hierarchical Deterministic Wallets 2 3

  3. https://github.com/bitcoin/bitcoin/blob/4b5659c6b115315c9fd2902b4edd4b960a5e066e/src/wallet/scriptpubkeyman.h#L52-L100

  4. Standards for Efficient Cryptography 1: Elliptic Curve Cryptography

  5. ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER)

  6. The AES-CBC Cipher Algorithm and Its Use with IPsec 2

  7. evp_sha512 - Linux man page

  8. The scrypt Password-Based Key Derivation Function

  9. Succinct Non-Interactive Zero Knowledge for a von Neumann Architecture, section 4.1: The PGHR protocol and the two elliptic curves

  10. ZIP-316: Unified Addresses and Unified Viewing Keys