Skip to main content

64 posts tagged with "ledger"

View All Tags

· 2 min read
Jared Corduan

High level summary

We made further progress on the conway ledger era. In particular, we expanded the ledger API significantly, including lots of governance features. We also made progress on the specification and corresponding work in the Haskell implementation.

We also continued to integrate the latest ledger packages into cardano node and addressed technical debt.

Low level summary

Expanded ledger API

The ledger API was significantly expanded to include:

  • a lot of protocol parameter support
  • versioning support (type level ledger eras and protocol versions)
  • auxiliary data support
  • many new lenses
  • support for witnesses
  • support for conway governance

See pull-3328.

Conway ledger rules

We have made progress on the formal ledger specification for the Conway era. Moreover, the corresponding Haskell updates were also completed:

Incremental SPO/DRep stake distribution computation

We have a working (and correct) proof of concept for how to use the incremental lambda calculus to maintain several of the stake distributions incrementally. For the per-SPO distribution, this is a performance improvement. For the (conway) per-DRep distribution, this is will allow those who have delegated their votes to a DRep to have time to react to any votes that they disapprove of. (Sorry, no code to share just yet, more to come.)

Technical debt

· 2 min read
Jared Corduan

High level summary

We have focused the last two weeks on CIP-1694, integration of the last several months of ledger work into consensus and node, and testing infrastrutcture that we will use in the conway ledger era.

Low level summary

Conway rules

We made a major update to the conway era so that the implementation is now in sync with the spec with respect to the ratification and enactment logic.

See the notes in pull-3291 for more details.

Constraint based generators

This week we hit a major milestone in our efforts to build out better property based testing support for the main ledger properties. The new constraint based generators can now generate full ledger states with what is probably very close to the real constraints (if anything, it is under constrained). Next we will work on generating a transaction in the context of a ledger state, which would allow us to actually start using these generators for real tests.

See pull-3219.

Preparing a release, now with proper versioning

After quite some time, we are ready to release a version of ledger that will work with a new version of consensus, using CHaPs.

See pull-3308.

Technical debt

  • We cleaned up the redeemer serialization code (to prevent future mistakes). See pull-3269.
  • We added a note to the Alonzo spec, specifying that the transaction inputs are lexicographically ordered in the Plutus script context. See pull-3306.
  • We fixed a problem with the address deserialiazation (we mistakenly fixed a bug in the Babbage era which cannot be fixed until Conway). See pull-3307.
  • We fixed a problem with our nix build. See pull-3311.
  • We fixed a problem with our NoThunks tests. See pull-3310.
  • We improved our nightly tests. See pull-3316.

· 3 min read
Jared Corduan

High level summary

This past two weeks saw many months worth of ledger changes integrated with the cardano-base and ouroboros-network repositories. The vast majority of the effort involved all the changes to the ledger serialization libraries (and the interplay with cardano-base) which now support proper versioning. Supporting the conway ledger era, and in particular allowing the ledger state to transition to a whole new system of governance, also played a noticeable part in the integration.

Besides the integration work, the team continues to address technical debt, improve the documentation, make our testing infrastructure better (such as experimenting with better nightly tests), and formally specifying more parts of CIP-1694.

Lower level summary

Integration work

Below is all the integration work completed, which will enable a release to node from the current ledger master. Many thanks to Alexey for this colossal undertaking!

Technical debt

Starting in major protocol version 9, zero-valued multi-assets will no longer be permitted in the CBOR specification. See pull-3241.

We now have our .cabal files being checked for a consistent formatting given by cabal format in our CI. See pull-3286.

We are still experimenting with better nightly tests for our long running property based tests. See pull-3276 and pull-3296.

Small documentation improvements

The hand proofs of the preservation ADA property have been added back to the Shelley ledger specification. See pull-3295.

We have clarified how the script integrity hash is computed in the CDDL description. See pull-3290.

Specifying CIP-1694

Our new formal specifications backed by Agda have seen a lot of progress! Upgrading to Agda 2.6.3 fixed our main build infrastructure problems, and we no longer have to rely on our custom fork. See pull-50.

Our progress on formally specifying CIP-1694 can be followed here:

Constraint based testing

We are still actively working on our new constraint based property testing framework. We have nearly all of the constraints for an entire ledger state hooked into generators, and tests that the generators do indeed obey the constraints. The variable count in the constraints is over 100! There is still much work to do on shrinking, making the generators faster, and writing actual property tests for the ledger, but the approach still seems viable and we are hopeful that it could replace our trace generators. The WIP can be followed here: pull-3219.

· 2 min read
Jared Corduan

High level summary

Much of the work the past two weeks involved integration efforts, cleaning up and debugging some serialization issues, adding tests, and work on large projects that are still ongoing. We also released a CIP this week that aims to make the ledger a registered CIP category.

Lower level summary

Ledger evolution CIP

We published a CIP that will make the Cardano ledger a registered category of the CIP process.

Serialization issues

We had previously thought that we had found a serialization problem with the redeemers, but it turned out to just be particularly confusing code. We have now clarified the issue for the future.

See pull-3263, pull-3269, and pull-3268.

Starting in protocol version 9, we will no longer accept duplicate keys in CBOR maps.

See pull-3277.

New tests

We added a new property test suite for some of our custom containers.

See pull-3270.

Progress on a better cost model serialization situation.

We are still working our way through issue-2902. Towards this end, we are now properly gating the new flexible encoders until version 9.

See pull-3274.

Nightly tests

We are still experimenting with moving more CI actions to GitHub actions.

See pull-3276.

Constraint based generators

We continue to add to our proof of concept for constraint based generators. See the previous ledger update for more information about this project.

· 3 min read
Jared Corduan

High level summary

The ledger team completed some preliminary ground work in preparation for CIP-1694 (restructuring the ledger state), fixed the PDF hosting problem (mostly the formal specs), built out more of the new user-friendly ledger API, finished a proof of concept for constraint-based generators for property tests (with the hopes of being able to replace our trace generators one day), and addressed technical debt.

Lower level summary

Restructuring the ledger state

The existing governance structures will be replace in the conway ledger era, as described in CIP-1694. In particular, the ledger rules will be restructured as follows:

   BBODY
|
|-------------------------------|
v v
TICK LEDGERS
| |
|---------| |
v v v
RUPD ~NEWEPOCH~ ~LEDGER~
| |
|----------| |--------|-------------------|-----------|
v v v v v
~EPOCH~ +ENACTMENT+ DELEGS UTXOW +TALLY+
| | |
|---------|------------| v |
v v v DELPL v
SNAP POOLREAP -UPEC- | UTXO
|--------| |
v v v
POOL DELEG ~UTXOS~

-..- Removed
+..+ Added
~..~ Modified

Moreover, the ledger state will also be restructured in accordance with the new rules. In the conway code, we have now removed UPEC, added TALLY, and stubbed ENACTMENT. We have also adapted all the data structures in the ledger state.

See:

PDF hosting

We now build all of our PDFs using a GitHub action which is triggered by pushing a tag with a specific form, cardano-ledger-spec-YYYY-MM-DD. The action creates a GitHub release containing the PDFs. The links in the main ledger README now point to the PDFs in the latest release.

See:

Powering the new ledger API

We have now removed all the HasField instances from the protocol parameter data types, and replaced them with lenses. This is probably the last major restructuring that the ledger team will do on the code base for the API for a while (the Plutus tools team will be working on it next, see here). We also added a new helpful function ensureMinCoinTxOut.

See:

Constraint-based generators

Our largest scale property tests generate an initial ledger state and a long sequence of valid blocks which span several epochs, mimicking a real network. These tests are, in theory, excellent for checking properties. They are, however, very difficult to maintain and are not as random as we would like (a lot of bias has to be introduced to keep the ledger state in enough order to keep generating blocks).

We would like to switch to tests which instead generate a random ledger state representative of not just an initial state, generate a single random valid block, and then test our properties. The hope is that these will be much more random and easier to maintain.

We have finished a proof of concept are encouraged that this approach could work!

See:

Technical debt

  • pull-3244 massive CI speedup
  • pull-3249 better types for fees in the protocol parameters
  • pull-3264 move our annotator code to the cardano-ledger-binary package where it belongs
  • pull-3239 move the Wdrls type to the Core module.