56 posts tagged with "ledger"

High level summary​

I am extremely excited to say that we now have a pull request up which introduces our new versioned CBOR serialization. This was an enormous effort, but it will solve a host of problems that we have had since the Shelley phase. It will take time to properly review it, and we will need to put in a lot of effort to integrate it with the downstream components, but this is a huge milestone. Additionally, we have a new CIP proposing a deprecation cycle for the transaction serialization schemes.

We also have a draft pull request that reworks how deposits are tracked. Users of the system will not notice any difference, but it is a necessary change needed to prepare the way for decentralizing the governance of Cardano.

Finally, we continued to address technical debt. In particular, we continued to make progress on bringing coherency and consistency to the code base with a common naming convention, and improving some error messages.

Lower level summary​

• We have a pull request up for our new versioned CBOR serialization. When we encounter a problem with our deserializers, it can be very difficult to implement a fix. It is difficult because we can only fix such issues during a hard fork, and leading up to the hard fork we must maintain two serializations for the same type in order to not cause unintended network splitting (the problematic version must be used before the hard fork, and the fixed version is used afterwards). This can be especially tricky with the FromCBOR typeclass, since it is not always easy to search for where all the problematic uses are located. The new versioned CBOR serialization allows us to gracefully handle this transition. See [pull-3138].
• We proposed a CIP for backwards compatibility of the transaction serialization schemes. See [pull-372].
• We have draft for the new deposit tracking. This draft is not as memory efficient as the final version will be, but it is a sufficient proof of concept that we can write property tests against, ensuring that we have not changed the semantics. We will optimize after we are sure of the correctness. See [pull-3127].
• We now provide better support for debugging failed Plutus scripts in an important helper function, named evaluateTransactionExecutionUnits. In particular, it now returns all the information needed to rerun the script with exactly the same arguments. This feature will end up appearing in the CLI and other tools from the Plutus tools team. See [pull-3135].
• We did a lot more renaming to bring coherency and consistency to the code base. See [pull-3126], [pull-3120], [pull-3118], and [pull-3116].
• We have added a few things to the ledger repository to make it conform to the Cardano Engineering Handbook See [pull-3139].

High level summary​

We have made the decision to use the formal ledger repository in place of a LaTeX spec for the next ledger era, and have added a lot of basic infrastructure to the model. In particular, we now have a lot of support for axiomatic set theory. While the next ledger era is still in the design phase, most of the team remains working on technical debt. In particular, we have moved a lot more code out of the Shelley specific modules and into a ledger core module, we have finished up our benchmarking around the problematic TICKF ledger transition (while improving the performance), made conveniences to the development environment, cleaned up all the recent changes to the cost model, added a lot of documentation, fixed some flaky tests, and deleted some dead code.

Lower level summary​

Axiomatic Set Theory​

The formal ledger model now has support for much of the set theory that we make use of in the formal ledger specifications. See [pull-20].

Completed Technical Debt​

• We have addressed issues with two of our most problematic and flaky tests. See [pull-3039] and [pull-3093].
• We have added more documentation and tests to the Twiddler module. This is a module which makes our CBOR serialization round-trip tests much more robust, and will also hopefully help enforce the mandate for downstream libraries to never re-serialize data that needs to be hashed. See [pull-3073] and [pull-3095] (we cannot merge 3095 just yet, due to a preference for merging other features).
• We have finished our long analysis of the problematic TICKF transition. We now have a lot of benchmarks surrounding this code, and have added performance improvements. See [pull-3068] and [issue-3035].
• We have restored support for ghcid in our repository. This is a tool for developing with Haskell that many of us find greatly improves our productivity by providing us with constant feedback from the type checker. See [pull-3112].
• After much activity on the cost model, we have done some final clean up of the code. See [pull-3075] and [pull-3101].
• We moved a lot of the existing user facing documentation regarding native tokens into the ledger repository, and cleaned it up (most of the heavy lifting was done by our amazing technical writers). See [pull-3091].
• We removed dead code. See [pull-3089].
• We moved a lot of code from the Shelley specific libraries to the ledger core library. See [pull-3109] and [pull-3110].
• We've removed more of the awkward legacy template Haskell names. See [pull-3108].

High level summary​

The ledger team is still primarily focused on addressing technical debt. We now have the infrastructure for versioning our serialization schemes, which we continue to put into action. We have made first steps towards getting proper support for the formal ledger repository (in particular, we've added nix builds and continuous integration support). We are wrapping up an investigation of the performance of a critical function used by the consensus layer for leader checks. Finally, we are improving the packaging and versioning of our code.

Lower level summary​

Completed Technical Debt​

• Because the Shelley ledger era was a complete re-write of the Byron ledger era, a lot of our code lives in the cardano-ledger-shelley package, though with hindsight we can say that much of it should live in cardano-ledger-core. We continue to move things to cardano-ledger-core, and have much more to come. [pull-3059]
• We now have the infrastructure to support versioned serialization schemes. The inability to do this has caused us a lot difficulties, such as [issue-3003], [issue-2965] and [issue-2444]. We are still in the process of switching to the versioned serialization scheme (such as [pull-3078]), but the infrastructure was completed in [pull-3063].
• We now have proper nix and CI support for the formal ledger project. [pull-19]
• A separate team is helping bring support for CHaP to all the cardano-node repositories. We have been helping out with this effort.

• A separate team continues working on upgrading all the cardano-node repositories to work with ghc 9.2.4. We have been helping out with this effort.

In-progress Technical Debt​

We also have several fairly large pull-requests in review that we are working on.

• In an on-going attempt to build out a more user-friendly API, we continue to remove HasField instances in place of using micro-lenses. The protocol parameters, in particular, are being worked on. [pull-3045]
• We are also renaming record fields to be consistent across the repository. [pull-3062]
• We are now cleaning up all the work we did to understand the performance of the TICKF transition. We have some improvements to the computatation as well. [pull-3068]
• We are adding more documentation, in particular to our Twiddler functionality. [pull-3073]
• The formal ledger is adding support for finite set theory. [pull-20]

Ledger Update​

We have continued focusing nearly entirely on addressing technical debt. A lot of design work has begun for the next ledger era, but we do not yet have anything concrete to share.

Technical debt issues completed​

• [issue-1676][pull-2992] We have finally removed the ledger dependency on the cardano-prelude package. It was barely used in the ledger repository, and it added a dependency that we did not want to maintain. It was a bit difficult to remove, and we had to coordinate removing it from cardano-base. A lot ended up going into pull-2992, due to the coordination effort, and we ended up updating Plutus as well. This means that we've now also made a lot of progress on the problematic cost model serialization issues described in issue-2902. In particular, after we resolve issue-3014, we will not have to wait an epoch before releasing a cost model for a new version of Plutus, as we had to do for the Vasil HF.
• [issue-3046][pull-3055] We moved a module that is now only used in Byron to a Byron package.
• [issue-3047][pull-3054] We improved the interface to the Value (multi-asset) type.
• [pull-3044] We debugged and fixed a tricky compilation issue. Certain kinds of field updates were adding approximately 20 minutes to our compile time!
• [issue-2932][pull-3036] As a part of our ongoing re-organization of the codebase, we have added a Cardano.Ledger.[Era].Core module to each ledger era that has a TxBody class. Most classes defined in the era should go in this new module. We also re-export the Cardno.Ledger.Core module and the previous Cardano.Ledger.[Era].Core modules from each era.

Technical debt in progress​

• [issue-3034][issue-3035][node-issue-4421] We are continuing to write benchmarks to understand exactly where all the time is being spent on executing the TICKF transition. The consolidation of the per-stake-credential stake distribution to the per-stake-pool distribution does seem to account for a large amount of time (near a second as written, which we have down to about half a second with some optimizations), but this does not account for everything. Applying the reward update may also be a big contributing factor.
• [pull-3033][pull-3038][pull-3041] A separate team is working on upgrading all the cardano-node repositories to work with ghc 9.2.4. We have been helping out with this effort.
• The nix scripts used to build our new formal ledger model do not work consistently for everyone, and we have been working on fixing these issues.
• [issue-3014] We are still working on adding a versioning scheme to all of the ledger serializers.

Ledger Update​

We have been focused nearly entirely on addressing technical debt.

• We introduced more consistent naming across eras, this time for the auxiliary data. See 3032.
• We made clear how the consumed functions differs between eras (which was a previous source of confusion), and added some related support to the fledgling ledger API. See 3016.
• We added clarity and organizational consistency to the main ledger era type synonyms. See 3017.
• We removed code duplication related to the input data hashes. See 3018.
• We split up a large module into smaller components. The large module was actually causing our CI to time out. See 3020.
• We cleaned up stale information in our cabal files, and upgraded cabal 3.8. See 3023, 3031, and 3028.
• We made consistent, standalone TxOut (transaction output) modules for every era. See 3024.
• We brought consistency to a maddening inconsistent use of type variables indicating the specific choice of cryptographic primitives. In particular, all uses of crypto have been renamed to c. See 3027.
• We did a clean up of the types in the Alonzo era. In particular, we switched to more parametric types that will compose better in the future and which simplifies the constraints. See 3029.
• We consolidated some existing fragmented logic regarding how we gather the scripts needed for a given transaction. This is a much needed cleanup to prevent future mistakes. See 3019.
• We fixed a problem with our generators that was causing a fair number of our property tests to fail in CI. See 3039.
• We have started the work to update Plutus. This will bring support for SECP in the next major protocol version, and also address a problem that we current have evolving the cost models. See 3030.
• We addressed a small issue that came up when integrating the conway era downstream, namely the lack of some serialization instances. See 3022.