Skip to main content

· 5 min read
Jared Corduan

High level summary​

The ledger team finished up the remaining work for tracking individual depots, built out the new Conway era transaction body (in line with CIP-1694), greatly reduce some problematically large calculations on the epoch boundary, and addressed technical debt.

Lower level summary​

Finishing the deposit tracking​

The initial work on the individual deposit tracking project focused only on correctness. As this is a large data structure (since its size is linear with respect to the number of registered stake credentials), it is very important that we also reduce the memory overhead as much as possible. Fortunately, we were able to add very little overhead for the deposits by using existing efficient data structures. The extra tracking now only incurs one word (8 bytes) per registered stake credential.

See:

New Conway era transaction​

We implemented the Conway era transaction body, which is in line with CIP-1694. Note that the Conway era implements, losing speaking, the parts of CIP-1694 that are not related to the liquid democracy (the "DReps"). The new transaction body adds the new governance actions and votes, while also deprecating the old governance structures (i.e. the old protocol parameter updates and MIR certificates).

We also now have the wire specification (CDDL file) and serialization code in place. The wire specification is still subject to change while we work on the Conway era, but it is now usable and has proper testing support (so that, for example, the serialization round-trips, etc).

See:

Optimizing the TICKF transition​

Every since the release of the Shelley era, we have been working to reduce the computational load placed on the node by the ledger at the epoch boundary. While still not perfect, we believe that we have removed one of the final problematically long epoch boundary computations that exacerbate situations like this. In particular, the problem involved the way in which the consensus layer obtains a view of the ledger for the purposes of checking the leadership schedule in a new epoch. We implemented a stopgap measure which now only incurs a single multi-second cost once per epoch instead of potentially several multi-second costs while the networks waits for the first block of a new epoch to be minted.

See:

Technical debt​

We closed the year out with a lot of reduction to the technical debt!

Improved ledger event​

  • pull-3212 - The ledger events are not guaranteed to appear in any given order within a block. For this reason, motivated by the use case in db-sync, the TotalDeposits event now includes a transaction ID and emits the change in deposits instead of the value.

Improved type saftey​

  • pull-3208 - We replaced NominalDiffTime with a newtype wrapper. The problem was that our CBOR encoders and decoders were using the wrong level of precision, having to due with with the Shelley genesis file. We removed the potential problem with a newtype wrapper.
  • pull-3167 - We now use a GADT to ensure consistency of the Plutus language in the types for TransactionScriptFailure and PlutusDebug.

Code/Module organization​

  • pull-3175 - The Allegra and Mary eras had an unusual relationship in our codebase, due to the uncertainly of release dates while we were implementing them. In particular, they were coupled in way that is different from the rest of the code base. With hindsight on our side, we split the combined shelley-ma Haskell package into two separate ledger era packages, which is now consistent with the rest of the repository and module structure.
  • pull-3184 - We created a core test sub-library, cleaning up a lot of our property test generator code.
  • pull-3210 - We moved the KeyPair type to the test library. Outside of testing, the ledger does not need to deal with signing keys, and since this is a topic that deserves the utmost care, it is best to make it clear that our use of signing keys is only for testing.
  • pull-3229 - We split the Cardano.Ledger.Alonzo.Data module, which is more consistent with the rest of the codebase.

Revert pointer address deprecation​

Thanks to one of our excellent internal auditors, @jmhrpr, we now have a better plan for deprecating pointer addresses. This meant that we had to revert the previous work to deprecate them.

See:

Miscellaneous​

  • pull-3205 - We removed deprecated type synonyms.
  • pull-3218 - We cleaned up the address deserialization.
  • pull-3223 - We fixed faulty address deserialization tests.
  • pull-3222 - We switched to a general type family TxOut from concrete ones, reducing many constraints.
  • pull-3224 - ShelleyGenesis is now parameterized by crypto instead of by era.
  • pull-3170 - We set the cabal-version to 3.0 in our projects.
  • pull-3172 - We removed the now useless EncodeMint/DecodeMint classes.
  • pull-3225 - We switch from ormolu to fourmolu. The reason was to be able to finally have more diff friendly code!

· 2 min read
Jordan Millar

2022-12-28 - 2023-01-10

High level summary​

Completed​

docs​

CI & project maintenance​

Developer experience​

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

In Progress​

CI & project mainteance​

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

· 2 min read
Sebastian Nagel

High-level summary​

This week, the Hydra team kicked off a new project together with Catalyst, the Cardano Foundation and researchers from IO: a proof of concept for voting on Hydra. The concluded work on validating the Head protocol using model-based testing and formulated next steps, fixed the transaction cost benchmark reporting on the website and reduced the cost for commit transactions by ~30% with the help of reference scripts.

The year is coming to an end and velocity will slow down a bit while the team focuses on wrapping up loose ends. The updates here will pause until January 2023.

What did the team achieve this week​

  • Kicked-off Hydra Voting project with Catalyst, CF, and IO Research.
  • Reduce commit transaction costs by ~30% with reference scripts.
  • Prepared an RFP for external audit of the Hydra Head solution.
  • Fixed transaction cost benchmarks for abort tx #631.
  • Recorded decision to use model-based testing (ADR22) and improved Model documentation.
    • Concluding the first increment on “Validate coordinated Head protocol” #194.
    • Formulated next step / follow-up on testing the Soundness property of our protocol #656.
  • Switched to using nix flakes for development setup and CI build #646.

What are the goals of next week​

  • Push ADR21 & tx validity gap over the finish line (smoke tests missing).
  • Integrate the hydra-tutorial.
  • Have a LaTeX write-up of the HeadV1 (Coordinated Hydra Head) spec.
  • Close & recap on the year with another monthly report (+ blog post).

· 2 min read
Jean-Philippe Raynaud

High level overview​

The Mithril team has released the new distribution 2248.1 of their nodes. They have published the first version of the Mithril cryptographic library on crates.io, the Rust community’s crate registry. They have implemented an optimization on the individual signatures that no longer embed the verification key and stake. They have also enhanced their testing strategy by implementing a workflow that tests that the client binaries produced for multiple platforms (Linux, MacOS and Windows) are able to verify and restore snapshots.

Finally, they have kept on simplifying the aggregator node's multi-signer by removing the signer registration and the certificate creation from its responsibilities.

Low level overview​

  • Implemented removing verification key and stake from single signatures #619
  • Completed the extraction of the signer registration from the multi-signer #642
  • Completed the extraction of the certificate creation from the multi-signer #638
  • Implemented a workflow to test client binaries (Linux / MacOS / Windows) #601
  • Completed the signature of the artifacts produced by the CI #587
  • Fixed the protocol parameters transition #627
  • Worked on optimizing the snapshot digest computation #510
  • Worked on enforcing the API protocol versions in the client and signer #633
  • Worked on deactivating the non certified signer registration mode #621
  • Worked on the re-genesis of the test networks #651

· 3 min read
Damian Nadales

High level summary​

During the past two weeks, the Consensus team finalized the QSM tests for the backing store and Mempool on the UTxO-HD branch with important discoveries regarding parallel QSM testing. We also worked with the Ledger team to envisage the modifications that are required in Ledger and Consensus to accommodate the changes in the crypto VRF and KES. The db-analyser now supports bechmarking the ledger operations, which will allow us to identify, debug, and profile potential performance problems. We drafted a document that defines how to manage the versions of Consensus-related packages. The top level documentation of ouroboros-network now features a description of the consensus components and provides a hyperlinked map to the modules documentation.

Workstreams​

UTxO HD prototype​

Whereas we had passing sequential state-machine tests for the mempool, the parallel case proved to be more challenging than we thought. The operation of adding a list of transactions to the mempool is not atomic and, as a result, when adding a list of transactions, transactions from other processes can be added in between. The mempool implementation handles this correctly, however this required us to redesign the parallel model we had to take the lack of atomicity into account.

Backing store property tests​

We finished refactoring the backing store property tests. The second review round is ongoing.

LSM tree implementation​

We are working on benchmarking (in terms of time and number of IO operations) fetching/looking up data from disk.

Genesis​

We worked on the design of a mechanism to prevent a DoS attack on our Genesis design related to rollbacks. This was arguably the biggest outstanding question.

During the discussions around Genesis, we noticed a design boundary that nicely delineates a fundamental component. We almost have a full Haskell prototype of it. It will be very nicely self-contained, perhaps even usable in the ultimate implementation!

New VRF and KES crypto integration​

We collaborated with the Ledger team on preparing the ledger state and crypto types to avoid huge allocation on the epoch boundary when changing aspects of the crypto that will only manifest in headers, not in the ledger states.

Technical debt​

We merged the pull-request that adds a support to db-analyser for benchmarking ledger operations. This will allow us to identify, debug, and profile potential performance problems. The benchmark focus on the main 5 ledger operations that are involved in chain syncing, block forging, and block validation, namely:

  1. Forecast.
  2. Header tick.
  3. Header application.
  4. Block tick.
  5. Block application.

The following figure shows a plot of the benchmarking results for the first 65 million blocks (approximately) of the Cardano chain. The thin yellow lines under the x-axis show the epoch boundaries, whereas the thick yellow lines correspond to the era transitions.

As we can see in this figure, era and epoch boundaries require more computation time. The ledger team are aware of this problem, and we are working to improve this situation.

Fostering collaboration​

We drafted a document motivating and defining how Consensus (and possibly other core teams) will/should manage our package versions. This pull-request garnered many great discussions from our team members and other teams too: Sebastian Nagel, Arnaud Bailly, Michael Peyton-Jones, Ziyang Liu, et al. We want to thank you all for your input, and we found this discussion very enlightening!

We merged the pull request that adds an overview of consensus to the top level documentation of ouroboros-network. This overview describes the consensus components and adds a hyperlinked map to the modules documentation.