Skip to main content

50 posts tagged with "ledger"

View All Tags

· 3 min read
Jared Corduan

High level summary​

The ledger team completed some preliminary ground work in preparation for CIP-1694 (restructuring the ledger state), fixed the PDF hosting problem (mostly the formal specs), built out more of the new user-friendly ledger API, finished a proof of concept for constraint-based generators for property tests (with the hopes of being able to replace our trace generators one day), and addressed technical debt.

Lower level summary​

Restructuring the ledger state​

The existing governance structures will be replace in the conway ledger era, as described in CIP-1694. In particular, the ledger rules will be restructured as follows:

   BBODY
    |
    |-------------------------------|
    v                               v
   TICK                            LEDGERS
    |                               |
    |---------|                     |
    v         v                     v
   RUPD     ~NEWEPOCH~            ~LEDGER~
              |                     |
              |----------|          |--------|-------------------|-----------|
              v          v                   v                   v           v
            ~EPOCH~     +ENACTMENT+         DELEGS              UTXOW       +TALLY+
              |                              |                   |
              |---------|------------|       v                   |
              v         v            v      DELPL                v
             SNAP      POOLREAP    -UPEC-    |                  UTXO
                                             |--------|          |
                                             v        v          v
                                            POOL     DELEG     ~UTXOS~

-..- Removed
+..+ Added
~..~ Modified

Moreover, the ledger state will also be restructured in accordance with the new rules. In the conway code, we have now removed UPEC, added TALLY, and stubbed ENACTMENT. We have also adapted all the data structures in the ledger state.

See:

PDF hosting​

We now build all of our PDFs using a GitHub action which is triggered by pushing a tag with a specific form, cardano-ledger-spec-YYYY-MM-DD. The action creates a GitHub release containing the PDFs. The links in the main ledger README now point to the PDFs in the latest release.

See:

Powering the new ledger API​

We have now removed all the HasField instances from the protocol parameter data types, and replaced them with lenses. This is probably the last major restructuring that the ledger team will do on the code base for the API for a while (the Plutus tools team will be working on it next, see here). We also added a new helpful function ensureMinCoinTxOut.

See:

Constraint-based generators​

Our largest scale property tests generate an initial ledger state and a long sequence of valid blocks which span several epochs, mimicking a real network. These tests are, in theory, excellent for checking properties. They are, however, very difficult to maintain and are not as random as we would like (a lot of bias has to be introduced to keep the ledger state in enough order to keep generating blocks).

We would like to switch to tests which instead generate a random ledger state representative of not just an initial state, generate a single random valid block, and then test our properties. The hope is that these will be much more random and easier to maintain.

We have finished a proof of concept are encouraged that this approach could work!

See:

Technical debt​

  • pull-3244 massive CI speedup
  • pull-3249 better types for fees in the protocol parameters
  • pull-3264 move our annotator code to the cardano-ledger-binary package where it belongs
  • pull-3239 move the Wdrls type to the Core module.

· 5 min read
Jared Corduan

High level summary​

The ledger team finished up the remaining work for tracking individual depots, built out the new Conway era transaction body (in line with CIP-1694), greatly reduce some problematically large calculations on the epoch boundary, and addressed technical debt.

Lower level summary​

Finishing the deposit tracking​

The initial work on the individual deposit tracking project focused only on correctness. As this is a large data structure (since its size is linear with respect to the number of registered stake credentials), it is very important that we also reduce the memory overhead as much as possible. Fortunately, we were able to add very little overhead for the deposits by using existing efficient data structures. The extra tracking now only incurs one word (8 bytes) per registered stake credential.

See:

New Conway era transaction​

We implemented the Conway era transaction body, which is in line with CIP-1694. Note that the Conway era implements, losing speaking, the parts of CIP-1694 that are not related to the liquid democracy (the "DReps"). The new transaction body adds the new governance actions and votes, while also deprecating the old governance structures (i.e. the old protocol parameter updates and MIR certificates).

We also now have the wire specification (CDDL file) and serialization code in place. The wire specification is still subject to change while we work on the Conway era, but it is now usable and has proper testing support (so that, for example, the serialization round-trips, etc).

See:

Optimizing the TICKF transition​

Every since the release of the Shelley era, we have been working to reduce the computational load placed on the node by the ledger at the epoch boundary. While still not perfect, we believe that we have removed one of the final problematically long epoch boundary computations that exacerbate situations like this. In particular, the problem involved the way in which the consensus layer obtains a view of the ledger for the purposes of checking the leadership schedule in a new epoch. We implemented a stopgap measure which now only incurs a single multi-second cost once per epoch instead of potentially several multi-second costs while the networks waits for the first block of a new epoch to be minted.

See:

Technical debt​

We closed the year out with a lot of reduction to the technical debt!

Improved ledger event​

  • pull-3212 - The ledger events are not guaranteed to appear in any given order within a block. For this reason, motivated by the use case in db-sync, the TotalDeposits event now includes a transaction ID and emits the change in deposits instead of the value.

Improved type saftey​

  • pull-3208 - We replaced NominalDiffTime with a newtype wrapper. The problem was that our CBOR encoders and decoders were using the wrong level of precision, having to due with with the Shelley genesis file. We removed the potential problem with a newtype wrapper.
  • pull-3167 - We now use a GADT to ensure consistency of the Plutus language in the types for TransactionScriptFailure and PlutusDebug.

Code/Module organization​

  • pull-3175 - The Allegra and Mary eras had an unusual relationship in our codebase, due to the uncertainly of release dates while we were implementing them. In particular, they were coupled in way that is different from the rest of the code base. With hindsight on our side, we split the combined shelley-ma Haskell package into two separate ledger era packages, which is now consistent with the rest of the repository and module structure.
  • pull-3184 - We created a core test sub-library, cleaning up a lot of our property test generator code.
  • pull-3210 - We moved the KeyPair type to the test library. Outside of testing, the ledger does not need to deal with signing keys, and since this is a topic that deserves the utmost care, it is best to make it clear that our use of signing keys is only for testing.
  • pull-3229 - We split the Cardano.Ledger.Alonzo.Data module, which is more consistent with the rest of the codebase.

Revert pointer address deprecation​

Thanks to one of our excellent internal auditors, @jmhrpr, we now have a better plan for deprecating pointer addresses. This meant that we had to revert the previous work to deprecate them.

See:

Miscellaneous​

  • pull-3205 - We removed deprecated type synonyms.
  • pull-3218 - We cleaned up the address deserialization.
  • pull-3223 - We fixed faulty address deserialization tests.
  • pull-3222 - We switched to a general type family TxOut from concrete ones, reducing many constraints.
  • pull-3224 - ShelleyGenesis is now parameterized by crypto instead of by era.
  • pull-3170 - We set the cabal-version to 3.0 in our projects.
  • pull-3172 - We removed the now useless EncodeMint/DecodeMint classes.
  • pull-3225 - We switch from ormolu to fourmolu. The reason was to be able to finally have more diff friendly code!

· 3 min read
Jared Corduan

High level summary​

The Plutus tools team at IOG has started helping the ledger team to build out a user friendly cardano-ledger-api package! A GitHub project will be filled out in the days ahead, people interested in the API can use it to follow along and join in on the conversations.

The ledger team has started using architectural decision records to leave a record of important decisions that the team makes. We will retroactively go back through past decisions and make ADRs for them.

The logic to track individual deposits is now nearly in place. We are prioritizing correctness with our first pull request, and will follow up with performance optimizations and general cleanup next.

Pointer addresses are being deprecated with the Cardano major protocol version 8. See CPS-0002 for more context.

Lower level summary​

Cardano ledger API​

The Plutus tools team has taken our minimal cardano-ledger-api package and started filling it out and adding much needed documentation. They have also added doctests! In the days to come, the Plutus tools team will map out a lot more work for the API and record it in this GitHub project.

See

Architectural Decision Records (ADRs)​

We are now providing more context and leaving a record of important decisions that are made in the ledger. The first ADR explains the very lightweight process.

See

Tracking individual deposits​

See ADR-3 for background. We now have the logic in place to track individual deposits, and a host of property tests to make sure that the logic is correct. The current implementation uses more memory than it needs to, and we will address that next, with our hope being to only use one word (8 bytes) per registered stake credential. There is a fair amount of other cleanup needed for general maintainability.

See

Removing pointer addresses​

Pointer addresses, which have never seen any real use (there are something like eleven on mainnet), are being deprecated starting at Cardano major version 9. CPS-0002 gives the context. We are disabling them by first preventing transaction outputs containing them from being serialized by the node at the moment we switch to version 9. At the hard fork after that, we will translate the existing few pointer addresses to enterprise addresses.

See

Technical debt​

  • pull-3162 - Sometimes we have to put safeguards in place for hard forks which may never be exercised. After we have passed the given hardfork, we are able to clean up the code and simplify our logic. We removed all of the ones that we are currently easily able to.
  • pull-3165 - We improved the type safety of our code while also discovering and fixing a serialization bug.
  • pull-3172 - We removed dead code.
  • pull-3175 - The Allegra and the Mary code used to be coupled in a particular way the we grew to dislike. We made these two ledger eras now uniform with the rest of our code base.
  • pull-3184 - We organized our property testing code.
  • pull-3200 - The Plutus tools teams fixed an outstanding bug in the translation from the ledger state to the Plutus script context.

· 3 min read
Jared Corduan

High level summary​

We released CIP-1694, our proposal for entering the Voltaire phase. Please come join the discussion, this will be an incredibly exciting transition for Cardano and we want everyone to participate!

We now have a sensible way to version all of the serialization schemes used in the ledger. The draft pull request was polished, reviewed, and merged this week. This solves many problems that have vexed us since the beginning of the Shelley ledger era.

Everyone working on the Cardano node is working together to improve our release process, and the ledger team in particular dedicated one engineer to help with these efforts for the next release.

Lower level summary​

The Conway ledger era​

The current proposal in CIP-1694 encompasses two new ledger eras. The first era will be called Conway, after the English mathematician John Horton Conway. The community facing aspects of the Conway ledger era will be very minimal, but it will pave the way for introducing liquid democracy. The details can be viewed here. We do not yet have a formal specification for the Conway era. Our plan is to debut the formal ledger model. Briefly, the Conway ledger era will:

  • introduce SPO voting for hard forks (in the spirit of the now abandoned CIP-47)
  • provide an on-chain mechanism for rotating the governance keys
  • re-plumb the ledger rules involving governance to be in line with CIP-1694

Versioned CBOR​

We now have the ability to easily tie our serialization schemes to the Cardano major protocol version. We still aim to preserve backwards compatibility as much as possible, but we now have a principled plan for resolving problems (see CIP-ledger-cbor). In particular, we can now address several long standing issues, such as issue-2444, issue-2965, and issue-3003.

The final (and massive!) pull request which brought us the versioning is pull-3138.

Deposit tracking​

The draft pull request which was exploring how best to track individual deposits is much closer now to being ready to take out of draft (pull-3127). For background on the issue, see issue-3113. This is quite an invasive change which effects many of our tests, which we are now addressing.

Technical debt​

As always, we keep working on technical debt. We have deduplicated a some things: pull-3129, pull-3162. We have memoized a problematic computation (though more due diligence is needed before we can merge): pull-3141.

Node release​

We have been helping with the node release efforts. See pull-4608.

· 3 min read
Jared Corduan

High level summary​

I am extremely excited to say that we now have a pull request up which introduces our new versioned CBOR serialization. This was an enormous effort, but it will solve a host of problems that we have had since the Shelley phase. It will take time to properly review it, and we will need to put in a lot of effort to integrate it with the downstream components, but this is a huge milestone. Additionally, we have a new CIP proposing a deprecation cycle for the transaction serialization schemes.

We also have a draft pull request that reworks how deposits are tracked. Users of the system will not notice any difference, but it is a necessary change needed to prepare the way for decentralizing the governance of Cardano.

Finally, we continued to address technical debt. In particular, we continued to make progress on bringing coherency and consistency to the code base with a common naming convention, and improving some error messages.

Lower level summary​

  • We have a pull request up for our new versioned CBOR serialization. When we encounter a problem with our deserializers, it can be very difficult to implement a fix. It is difficult because we can only fix such issues during a hard fork, and leading up to the hard fork we must maintain two serializations for the same type in order to not cause unintended network splitting (the problematic version must be used before the hard fork, and the fixed version is used afterwards). This can be especially tricky with the FromCBOR typeclass, since it is not always easy to search for where all the problematic uses are located. The new versioned CBOR serialization allows us to gracefully handle this transition. See [pull-3138].
  • We proposed a CIP for backwards compatibility of the transaction serialization schemes. See [pull-372].
  • We have draft for the new deposit tracking. This draft is not as memory efficient as the final version will be, but it is a sufficient proof of concept that we can write property tests against, ensuring that we have not changed the semantics. We will optimize after we are sure of the correctness. See [pull-3127].
  • We now provide better support for debugging failed Plutus scripts in an important helper function, named evaluateTransactionExecutionUnits. In particular, it now returns all the information needed to rerun the script with exactly the same arguments. This feature will end up appearing in the CLI and other tools from the Plutus tools team. See [pull-3135].
  • We did a lot more renaming to bring coherency and consistency to the code base. See [pull-3126], [pull-3120], [pull-3118], and [pull-3116].
  • We have added a few things to the ledger repository to make it conform to the Cardano Engineering Handbook See [pull-3139].