Skip to content

Commit ae95fa5

Browse files
nfrisbynfrisby
committed
ProtocolVersioning.md: first draft
1 parent 35cc0cb commit ae95fa5

File tree

2 files changed

+155
-0
lines changed

2 files changed

+155
-0
lines changed

‎docs/website/contents/for-developers/ProtocolVersioning.md

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Introduction
2+
3+
The Cardano protocol will change over time.
4+
The community innovates applications and updates feature priorities, researchers refine Ouroboros, and developers improve the code.
5+
6+
This document describes how Cardano accomodates these changes, as a community, as a software engineering project, and as a distributed system.
7+
8+
## The Protocol Version
9+
10+
Each change to the Cardano protocol advances through the following steps before influencing the network's chain.
11+
12+
- Someone (you!) specifies a change that is backwards-incompatible but worthwhile.
13+
For example, it could be a [Cardano Improvement Proposal](https://cips.cardano.org/) (CIP).
14+
- The Technical Steering Committee and Ouroboros researchers address the worth, safety, and future consequences of the proposed solution.
15+
- Cardano developers prepare a software release that includes the changes but only enables for chains on which the protocol version has been incremented.
16+
- The community uses on-chain governance to increment the protocol version, thereby adopting the new rules.
17+
(Stake pool operators must not vote to increment the protocol version before updating their software!)
18+
19+
Cardano philosophy mandates that the community has the final say regarding if and when to adopt some protocol change.
20+
They accordingly control the final step above: the community increments the protocol version on mainnet, not the developers.
21+
22+
It is crucial that the community does not increment the protocol version before enough nodes have upgraded their software.
23+
Each protocol version increment corresponds to a _hard fork_, because the changes are backwards-incompatible: nodes running the old code will not be able to adopt blocks minted by the new code.
24+
Even if some block content happened to be valid in both the old and the new protocol, each software release explicitly refuses any block that (extends a ledger state that) has a protocol version greater than that release was intended to implement — see the MaxMajorPV constant.
25+
26+
Even software released well after mainnet increments the protocol version must implement the protocol both with and without that increment's changes.
27+
The Cardano node must correctly process blocks with older protocol versions, eg when catching up to mainnet for the first time or after being offline for a long time.
28+
So every protocol change must remain conditional within all future releases, instead of only within the particular release that mainnet used for that increment of the protocol version.
29+
(Exceptions are possible if the community decides to somehow truncate/compress a prefix of the historical chain.)
30+
31+
## When to Increment the Protocol Version?
32+
33+
Fundamentally, the protocol version must be incremented if some change to the protocol that an old node and a new node that received the same messages otherwise might disagree on which block is the best valid block, aka maintain consensus.
34+
A more precise and actionable answer will be given after introducing more context.
35+
36+
### The Abstraction Ladder
37+
38+
For the sake of this discussion, Cardano fixes a ladder of abstractions between the following two extrema.
39+
40+
- *Opacity* (bottom rung).
41+
The Cardano node exchanges opaque bytestrings with peers in the network and with local users (eg wallets).
42+
At any given time, the node can determine whether such a bytestring is valid, and that predicate depends on past bytestrings.
43+
44+
- *Transparency* (top rung).
45+
Users interact with Cardano nodes to interpret the current state of the network and to evolve it according to their goals.
46+
47+
An analogous spectrum applies to any useful computer system, and there are longstanding traditions of how to organize the rungs between these two.
48+
The Cardano node is internally organized as the following ladder.
49+
50+
- Opaque Rung (bottom).
51+
- CBOR Rung.
52+
- The _generic data model_ of [RFC 8949 Concise Binary Object Representation](https://cbor.io/).
53+
- Cardano Message Data Model Rung.
54+
- A data model for Cardano messages that is sufficiently abstract for Cardano developers to easily interpret them in a consistent and useful way (eg algebraic data types).
55+
- Note that these messages include transaction IDs, transactions, block headers, blocks, queries, query replies, etc, but not the node's internal state [^state-data-model].
56+
- Ideally, every use case is satisfied by exchanging messages with the node.
57+
- Some fields within this model are hashes computed in terms of the CBOR Rung.
58+
- Cardano Consensus and Ledger Rung.
59+
- Received messages must be validated and invalid messages must not be sent.
60+
In practice, this requires the node to maintain as state a summary of all relevant messages received that is sufficient to determine which messages to send (eg how to respond to queries).
61+
- All validity predicates are defined in terms of the Cardano Message Data Model Rung.
62+
- When to mint a new block, which chain it should extend, and which transactions it should include.
63+
- Cardano Networking Rung.
64+
- Maintaining a robust connection to the rest of the Cardano network that keeps throughput high and latency low.
65+
- Cardano Executable Rung.
66+
- [The Cardano node](https://github.com/IntersectMBO/cardano-node)'s configuration interface, the [Cardano Command-Line Interface](https://github.com/IntersectMBO/cardano-cli), etc.
67+
- This rung also contains alternatives to that CLI and/or extensions of it developed by the broader community, including wallets, indexers, Mithril, decentralized applications (aka dapps), Layer 2, etc.
68+
- Transparent Rung (top).
69+
70+
### The Message Data Model Onion
71+
72+
The Cardano Message Data Model Rung is organized as the layers of an onion.
73+
Some are some other rungs, but their structure is beyond the intended scope of this document.
74+
75+
- [Ledger Layer](https://github.com/IntersectMBO/cardano-ledger/).
76+
- The notion of _era_ is an organizing concept within the Ledger Layer --- more on this below.
77+
- For each era, the content of its blocks, transactions, almost all queries, and almost all query responses.
78+
- In an Ouroboros Leios implementation, also input blocks and endorsement blocks.
79+
- [Consensus Layer](https://github.com/IntersectMBO/ouroboros-consensus/).
80+
- The content of the few queries and their responses that either do not depend on the ledger era or else regard when prior era transitions happened, which the Ledger Layer does not record.
81+
- An additional envelope wrapping block headers, blocks, transactions, almost all queries, and almost all query responses that indicates a particular era.
82+
The era tag behaves as an additional field [^era-tag-parse] of block header or a transaction, which affects the validity but not the identifying hash.
83+
- In an Ouroboros Peras implementation, also votes and certificates.
84+
- [Network Layer](https://github.com/IntersectMBO/ouroboros-network) (outermost) [^mux-layer].
85+
- ChainSync, BlockFetch, TxSubmission, LocalStateQuery, etc; see "Chapter 3 Mini Protocols" within the [Ouroboros Network Specification](https://ouroboros-network.cardano.intersectmbo.org/pdfs/network-spec/network-spec.pdf) [^network-pdf-link].
86+
- The content of each mini protocol message, excluding the payload.
87+
- In an Ouroboros Peras and/or Ouroboros Leios implementation, additional mini protocols.
88+
89+
### Precise Answer
90+
91+
The protocol version must be incremented if an old node and a new node would otherwise disagree on any of the following.
92+
93+
- *Chain Validity Predicates*, the validity predicates for headers, blocks, and/or transactions within the Cardano Consensus and Ledger Rung.
94+
As of Peras, this would also include votes and certificates.
95+
As of Leios, this would also include input blocks and endorsement blocks.
96+
- *Chain Order*, the [partial order](https://en.wikipedia.org/wiki/Partially_ordered_set) that determines one chain is better than another within the Cardano Consensus and Ledger Rung.
97+
Tiebreakers are a gray area, since Praos does not inherently rely on them.
98+
As of Peras, this would depend on voting quorums/certificates.
99+
- *Chain Data Model*, the parts of the Cardano Message Data Model Rung used by the Chain Validity Predicates and the Chain Order.
100+
- A change to the data model of the node's maintained state --- as opposed to its messages --- would not inherently trigger this.
101+
- However, if such a change were meaningful (ie not just a refactor/optimization/etc), then some Chain Validity Predicates would necessarily also change.
102+
- *Chain Codecs*, the parts of the mapping from the CBOR Rung to the Chain Data Model that affect the hashes therein.
103+
Two nodes won't even be able to exchange transactions, headers, and/or blocks if they disagree on their identifying hashes.
104+
105+
The most common change in Cardano's protocol so far has been the addition of a new kind/variant of transaction, which triggers all of these except the Chain Order.
106+
The Chain Order has not yet changed --- it's always been length.
107+
On the other hand, the tiebreaker has changed, and out of caution those changes were bundled up with unrelated changes that independently required a protocol version increment.
108+
109+
It's also worth noting that the Consensus Layer and Network Layer or the Cardano Message Data Model Rung can change without without incrementing the protocol version.
110+
The most common example that queries can be changed without triggering any of the above (queries do not contain any of the Chain Data Model).
111+
Mini protocol changes are similar, except for their Chain Data Model payloads.
112+
Such changes are conditional on the per-connection version negotiated by the Handshake mini protocol instead of the protocol version.
113+
114+
## Ledger Eras
115+
116+
Any Cardano node implemented in a strongly-typed language is likely going to declare distinct types for Chain Data Models that differ between protocol versions; more precise types yield stronger assurances from the typechecker.
117+
For the sake of such implementations, the ledger introduces a new era whenever the Chain Data Model changes.
118+
The [sequence of eras](https://github.com/cardano-foundation/CIPs/blob/master/CIP-0059/feature-table.md) at the time of writing are: Byron [^byron-halves], Shelley, Allegra, Mary, Alonzo, Babbage, and Conway.
119+
120+
A Cardano node implementation or other tooling implemented in a language prone to dynamic typing might find it convenient to ignore the era tags for the most part.
121+
However, the era tags must still be validated according to the intended function from Cardano protocol versions to Ledger eras, since a strongly-typed node would fail to accept a block with the wrong era tag.
122+
123+
Protocol changes that do not also introdue a new era are known as "intra-era hard forks"
124+
These are certainly sound if they only change the Chain Validity Predicate, but they generally could do more.
125+
So far, Cardano has forbidden intra-era hard forks from changing the Chain Codecs.
126+
This could be allowed in either of two ways.
127+
128+
- The Consensus envelope could store the protocol version instead of the era tag.
129+
At that point, the era tag would be an internal implementation detail of strongly-typed node implementations, and could even freely differ between them.
130+
The only downside is that the Consensus envelope's byte size might grow slightly, but it'll still be dwarfed by the Ledger Layer portion of the wrapped header/block/transaction.
131+
The code would also need to store the mapping from protocol version to era tags, but that is known at compile-time --- there is the slight complication that it might vary between chains (eg testnets), but they already tend to have different configuration files.
132+
- The Consensus envelope could be entirely eliminated if the object's slot could be always be determined even without knowing the protocol version.
133+
The slot could then be used to determine the appropriate protocol version, based on the node's state.
134+
The downside here is significant: programs that do not maintain the node's state (since that is costly to do) would not necessarily be able to parse headers, blocks, and transactions, since they wouldn't necessarily know the mapping from slot to protocol version (since it's determined by on-chain governance).
135+
136+
It might be plausible for the second option to restrict stateless programs to assume they're downstream of a node that had annotated the blocks and/or transactions with the correct protocol version.
137+
At that point, though, this option is basically the same as the first, except that nodes would exclude the Consensus envelope when exchanging with other nodes.
138+
139+
[^state-data-model]: *Remark*.
140+
The state maintained by the Cardano Consensus and Ledger Rung also has a data model, but, in contrast to messages, it is an internal implementation detail of the node.
141+
Different node implementations could have different state data models, but still implement the same protocol, ie send/receive/accept/reject the exact same messages.
142+
143+
[^era-tag-parse]: *Remark*.
144+
In particular, it's the only part of the block or transaction whose codecs does not depend on the era.
145+
146+
[^mux-layer]: *Disclaimer*.
147+
This Network Layer of the Cardano Message Data Model Onion actually contains an additional layer, the Mux Layer, but that is transparent to the rest of system and beyond the intended scope of this doucment.
148+
149+
[^network-pdf-link]: *Note*.
150+
If that link breaks, the document can usually be obtained via the "The Shelley Networking Protocol" hyperlink in the `README` file of the [`ouroboros-network` repository](https://github.com/IntersectMBO/ouroboros-network).
151+
152+
[^byron-halves]: *Historical Note*.
153+
Byron originally had two halves (Ouroboros Classic with Epoch Boundary Blocks versus round-robin with no EBBs), but today's node does not differentiate the two.
154+
It uses the same Chain Validity Predicate for both: the Permissive BFT protocol, which essentially ignores any EBBs.

‎docs/website/sidebars.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ const sidebars = {
4040
'for-developers/CivicTime',
4141
'for-developers/AbstractProtocol',
4242
'for-developers/HardForks',
43+
'for-developers/ProtoclVersioning',
4344
'for-developers/AddingAnEra',
4445
'for-developers/ChainSync',
4546
'for-developers/HardWonWisdom',

0 commit comments

Comments
 (0)