Ledger Quarterly Update
2023-01 - 2023-03
Main achievements
CIPs
- Entering the Voltaire phase - CIP-1694 received a major update after participation in the design has expanded to more and more people, including those who attended the Colorado workshop. See CIP-1694.
- Ledger CIP category - The ledger team continues to embrace the CIP process, and has begun the process of registering the ledger as an official CIP category. See CIP-84.
- Ledger serialization - A CIP for the ledger serialization deprecation cycle has been accepted. See CIP-80.
Formal ledger model
Our new formal specifications backed by Agda have seen a lot of progress. The majority of the ideas in CIP-1694 are now present, and we have made enough progress that we can now safely say that the PDF produced by the Agda model will be the official ledger specification for the Conway ledger era. See the repository.
Conway ledger era
Progress on the Haskell implementation of CIP-1694 has gone hand in hand with the formal model. The major component still missing is the DRep stake distribution, which still presents some technical challenges.
[pull-3176] [pull-3216] [pull-3226] [pull-3291] [pull-3326] [pull-3330] [pull-3339]
DRep stake distribution computation
Adding another large stake distribution to the ledger state must proceed with caution. We do not want the memory used by the node to increase too much, and performance problems can lead to reduced block production. We have prototyped, tested, and benchmarked several approaches that could give us the current DRep stake distribution at each epoch boundary. This has very important implications, since we want every ADA holder to be able to at any time (such as during a contentious vote) register themselves as a DRep and still have time to vote themselves on the issue.
[pull-3344] [pull-3353] [pull-3364]
Integration work
The ledger has made some wonderful improvements over the past six months, but which entail a significant amount of integration efforts:
- Our new versioned CBOR schemes
- Individual deposit tracking
- An improved cross-era interface utilizing lenses
- A new ledger API
- Re-arranging the ledger stake in preparation for CIP-1694
- Versioning our Haskell packages using CHaPs.
- Consistent conventions for variable names
[pull-3279] [pull-3282] [pull-3288] [pull-3289] [pull-3292] [pull-3297] [pull-3298] [pull-3299] [pull-3300] [pull-3302] [pull-3303] [pull-3308] [pull-3342] [pull-3345] [pull-3356] [pull-3357] [pull-3360] [pull-3361] [pull-3363] [pull-4349] [pull-378] [pull-376] [pull-373] [pull-370] [pull-361] [pull-4976] [pull-5013]
Deposit tracking
Individual deposits (for stake credential and stake pool registrations) were not tracked by the ledger. Deposits were returned according to the current protocol parameters. When the values of these two protocol parameters change, the deposit pot is adjusted by adding to, or removing from, the reserves.
This has several problems:
- Most people expect a deposit to be paid back exactly.
- We cannot increase the deposit amount once the reserves hits zero.
- If it becomes known that the deposit amount is going to be increased, free Lovelace can be earned by registering credentials.
- Because of the problems above, it is going to be incredibly hard to ever change the values.
- There is a serious issue involving hard forks. The consensus layer makes the decision about whether or not to enact a hard fork based on the protocol parameter update state two stability windows before the end of the epoch. However, the ledger will reject a protocol parameter update on the epoch boundary if the deposit pot adjustments cannot be reconciled with the reseve pot. This means that if quorum is met regarding changing the major protocol version, but the update is rejected on the epoch boundary, consensus will change the era but the ledger will not change the major protocol version, leaving the ledger in a split-brain state.
Because we never actually changed the values of the two deposits amounts in the protocol parameters on mainnet, we were able to retroactively change the behavior. We made the following changes:
- Individual deposits are tracked in the
DState. - The amount deposited is always returned.
[pull-3195] [pull-3202] [pull-3217]
New ledger API
We have significantly built up the ledger API.
We will eventually replace much of the cardano-api in the node repository with this ledger API.
[pull-3242] [pull-3248] [pull-3328]
Constraint-based generators
Our largest scale property tests generate an initial ledger state and a long sequence of valid blocks which span several epochs, mimicking a real network. These tests are, in theory, excellent for checking properties. They are, however, very difficult to maintain and are not as random as we would like (a lot of bias has to be introduced to keep the ledger state in enough order to keep generating blocks).
We have a new declaritive infrastructure for building constraint-based generators, which instead generate a random ledger state representative of not just an initial state, but also those representative of the end result of a long sequence of valid blocks. Moreover, these generators are very fast and are much more random than our old generators. Before we can start using them for our existing property tests, however, we still need to expand them to generate a valid block for a given ledger state.
Technical debt
We continued to address technical debt as much as we can.
[pull-3167] [pull-3170] [pull-3172] [pull-3175] [pull-3184] [pull-3205] [pull-3208] [pull-3210] [pull-3212] [pull-3218] [pull-3222] [pull-3223] [pull-3224] [pull-3225] [pull-3229] [pull-3239] [pull-3241] [pull-3244] [pull-3245] [pull-3249] [pull-3260] [pull-3263] [pull-3264] [pull-3268] [pull-3269] [pull-3270] [pull-3274] [pull-3276] [pull-3277] [pull-3286] [pull-3290] [pull-3295] [pull-3296] [pull-3306] [pull-3307] [pull-3310] [pull-3311] [pull-3316] [pull-3320] [pull-3323] [pull-3327] [pull-3331] [pull-3332] [pull-3333] [pull-3338] [pull-3341] [pull-3347] [pull-3350] [pull-3351] [pull-3352] [pull-3354]
Critical fixes
We fixed two critical issues:
- Growing block production delay on the epoch boundary: [pull-3209]
- Unexpected node shutdown from
balanceR: [pull-3343]
Next steps
- Conway spec - Complete the first version of the conway formal specification.
- DRep stake distribution - Have the ledger compute the DRep stake distribution with acceptible performance.
- Devnet ready - Have the Haskell implementation of the conway era in sync with the formal specification, and integrate the changes with consensus and node. All the details might not be finalized, but the wire specification and the API should be stable so that conway can be placed on a devnet for tool builders to start integrating with.
- Plutus V3 - Integrate Plutus V3 into the ledger, including a new script context which supports DReps.
More details
This quarterly report was based off of the following fortnightly ones: