50 posts tagged with "ledger"

High level summary​

We have made the decision to use the formal ledger repository in place of a LaTeX spec for the next ledger era, and have added a lot of basic infrastructure to the model. In particular, we now have a lot of support for axiomatic set theory. While the next ledger era is still in the design phase, most of the team remains working on technical debt. In particular, we have moved a lot more code out of the Shelley specific modules and into a ledger core module, we have finished up our benchmarking around the problematic TICKF ledger transition (while improving the performance), made conveniences to the development environment, cleaned up all the recent changes to the cost model, added a lot of documentation, fixed some flaky tests, and deleted some dead code.

Lower level summary​

Axiomatic Set Theory​

The formal ledger model now has support for much of the set theory that we make use of in the formal ledger specifications. See [pull-20].

Completed Technical Debt​

• We have addressed issues with two of our most problematic and flaky tests. See [pull-3039] and [pull-3093].
• We have added more documentation and tests to the Twiddler module. This is a module which makes our CBOR serialization round-trip tests much more robust, and will also hopefully help enforce the mandate for downstream libraries to never re-serialize data that needs to be hashed. See [pull-3073] and [pull-3095] (we cannot merge 3095 just yet, due to a preference for merging other features).
• We have finished our long analysis of the problematic TICKF transition. We now have a lot of benchmarks surrounding this code, and have added performance improvements. See [pull-3068] and [issue-3035].
• We have restored support for ghcid in our repository. This is a tool for developing with Haskell that many of us find greatly improves our productivity by providing us with constant feedback from the type checker. See [pull-3112].
• After much activity on the cost model, we have done some final clean up of the code. See [pull-3075] and [pull-3101].
• We moved a lot of the existing user facing documentation regarding native tokens into the ledger repository, and cleaned it up (most of the heavy lifting was done by our amazing technical writers). See [pull-3091].
• We removed dead code. See [pull-3089].
• We moved a lot of code from the Shelley specific libraries to the ledger core library. See [pull-3109] and [pull-3110].
• We've removed more of the awkward legacy template Haskell names. See [pull-3108].

High level summary​

The ledger team is still primarily focused on addressing technical debt. We now have the infrastructure for versioning our serialization schemes, which we continue to put into action. We have made first steps towards getting proper support for the formal ledger repository (in particular, we've added nix builds and continuous integration support). We are wrapping up an investigation of the performance of a critical function used by the consensus layer for leader checks. Finally, we are improving the packaging and versioning of our code.

Lower level summary​

Completed Technical Debt​

• Because the Shelley ledger era was a complete re-write of the Byron ledger era, a lot of our code lives in the cardano-ledger-shelley package, though with hindsight we can say that much of it should live in cardano-ledger-core. We continue to move things to cardano-ledger-core, and have much more to come. [pull-3059]
• We now have the infrastructure to support versioned serialization schemes. The inability to do this has caused us a lot difficulties, such as [issue-3003], [issue-2965] and [issue-2444]. We are still in the process of switching to the versioned serialization scheme (such as [pull-3078]), but the infrastructure was completed in [pull-3063].
• We now have proper nix and CI support for the formal ledger project. [pull-19]
• A separate team is helping bring support for CHaP to all the cardano-node repositories. We have been helping out with this effort.

• A separate team continues working on upgrading all the cardano-node repositories to work with ghc 9.2.4. We have been helping out with this effort.

In-progress Technical Debt​

We also have several fairly large pull-requests in review that we are working on.

• In an on-going attempt to build out a more user-friendly API, we continue to remove HasField instances in place of using micro-lenses. The protocol parameters, in particular, are being worked on. [pull-3045]
• We are also renaming record fields to be consistent across the repository. [pull-3062]
• We are now cleaning up all the work we did to understand the performance of the TICKF transition. We have some improvements to the computatation as well. [pull-3068]
• We are adding more documentation, in particular to our Twiddler functionality. [pull-3073]
• The formal ledger is adding support for finite set theory. [pull-20]

Ledger Update​

We have continued focusing nearly entirely on addressing technical debt. A lot of design work has begun for the next ledger era, but we do not yet have anything concrete to share.

Technical debt issues completed​

• [issue-1676][pull-2992] We have finally removed the ledger dependency on the cardano-prelude package. It was barely used in the ledger repository, and it added a dependency that we did not want to maintain. It was a bit difficult to remove, and we had to coordinate removing it from cardano-base. A lot ended up going into pull-2992, due to the coordination effort, and we ended up updating Plutus as well. This means that we've now also made a lot of progress on the problematic cost model serialization issues described in issue-2902. In particular, after we resolve issue-3014, we will not have to wait an epoch before releasing a cost model for a new version of Plutus, as we had to do for the Vasil HF.
• [issue-3046][pull-3055] We moved a module that is now only used in Byron to a Byron package.
• [issue-3047][pull-3054] We improved the interface to the Value (multi-asset) type.
• [pull-3044] We debugged and fixed a tricky compilation issue. Certain kinds of field updates were adding approximately 20 minutes to our compile time!
• [issue-2932][pull-3036] As a part of our ongoing re-organization of the codebase, we have added a Cardano.Ledger.[Era].Core module to each ledger era that has a TxBody class. Most classes defined in the era should go in this new module. We also re-export the Cardno.Ledger.Core module and the previous Cardano.Ledger.[Era].Core modules from each era.

Technical debt in progress​

• [issue-3034][issue-3035][node-issue-4421] We are continuing to write benchmarks to understand exactly where all the time is being spent on executing the TICKF transition. The consolidation of the per-stake-credential stake distribution to the per-stake-pool distribution does seem to account for a large amount of time (near a second as written, which we have down to about half a second with some optimizations), but this does not account for everything. Applying the reward update may also be a big contributing factor.
• [pull-3033][pull-3038][pull-3041] A separate team is working on upgrading all the cardano-node repositories to work with ghc 9.2.4. We have been helping out with this effort.
• The nix scripts used to build our new formal ledger model do not work consistently for everyone, and we have been working on fixing these issues.
• [issue-3014] We are still working on adding a versioning scheme to all of the ledger serializers.

Ledger Update​

We have been focused nearly entirely on addressing technical debt.

• We introduced more consistent naming across eras, this time for the auxiliary data. See 3032.
• We made clear how the consumed functions differs between eras (which was a previous source of confusion), and added some related support to the fledgling ledger API. See 3016.
• We added clarity and organizational consistency to the main ledger era type synonyms. See 3017.
• We removed code duplication related to the input data hashes. See 3018.
• We split up a large module into smaller components. The large module was actually causing our CI to time out. See 3020.
• We cleaned up stale information in our cabal files, and upgraded cabal 3.8. See 3023, 3031, and 3028.
• We made consistent, standalone TxOut (transaction output) modules for every era. See 3024.
• We brought consistency to a maddening inconsistent use of type variables indicating the specific choice of cryptographic primitives. In particular, all uses of crypto have been renamed to c. See 3027.
• We did a clean up of the types in the Alonzo era. In particular, we switched to more parametric types that will compose better in the future and which simplifies the constraints. See 3029.
• We consolidated some existing fragmented logic regarding how we gather the scripts needed for a given transaction. This is a much needed cleanup to prevent future mistakes. See 3019.
• We fixed a problem with our generators that was causing a fair number of our property tests to fail in CI. See 3039.
• We have started the work to update Plutus. This will bring support for SECP in the next major protocol version, and also address a problem that we current have evolving the cost models. See 3030.
• We addressed a small issue that came up when integrating the conway era downstream, namely the lack of some serialization instances. See 3022.

Ledger Update​

Since finishing up support for the Vasil Hardfork, the ledger team has been focused on two main things: a new ledger era and technical debt.

New minimal ledger era​

We have implemented a new ledger era named conway which is nearly identical to the babbage era. This has been the first time that we have been able to see what a minimal ledger era looks like. We have finished this task, modulo any integration issues that might come up. The only thing that the conway era does differently from the babbage era is provide support for rotating the master keys using the hardfork combinator's state translation. We may end up adding features to the conway era, but it is a nice exercise seeing what it looks like to get a minimal ledger era supported in all the downstream components.

Addressing technical debt​

We have been addressing technical debt, mostly in an effort to make the repository a more friendly code base to work in.

• We have begun work on a ledger API, called cardano-ledger-api.
• We have done a big re-design of the major type classes used in the ledger. With hindsight on our side, we now have something much more organized and easier to use.
• We have done a lot of re-naming. The names across eras are now much more uniform, avoid certain confusions that plagued us, and are clearer in where they are from.
• We have reduced a lot of code duplication that could lead to bugs if you do not have the whole code base in your head.
• We have added a handful of performance improvements.
• We added type safety in a number of locations. In particular, the type of values that can be minted in a transaction no longer allow for Lovelace in the type, and some functions which used to handle both timelock scripts and plutus script now correctly enoforce at the type level that only one of them can be used.
• We made our generators so that they now produce a much richer set of valid serializations. There is room within CBOR to serialize the same data structure in multiple ways, and it is helpful to have the generators use a wide variety.
• We have begun re-organizing our test suites.