High-level summary​

This week, the Hydra team addressed issues with committing complex UTxOs into a Hydra Head, prepared the hydra-node to be run on mainnet (soon) and improved the test suite of the hydra-plutus scripts. The specification is constantly getting improved upon reviewer feedback on overleaf - this week the graphical notation was refined to be more transaction-focused.

What did the team achieve this week​

• Clarified “bug” of committing UTxOs with reference scripts to a Hydra head #737
• Prepare usage of hydra-node on mainnet by updating command line options #715
• Improved our mutation test suite to all expected errors #705
• Re-opened our persistent hydraw demo head on preprod using version 0.9.0
• Updated on-chain graphs to be more transaction-focused in the specification

What are the goals of next week​

• Complete mainnet compatibility feature
• Prepare the Hydra workshop
• Address all todo’s in the Hydra specification
• Unblock the auctions project with “commit from script” a solution

High level overview​

The Mithril team worked on deploying the era activation mechanism to their test networks and preparing the upgrade procedure to be implemented on the signer nodes run by the SPOs. They also refactored the state machines of the aggregator and signer nodes so that they can better handle critical errors, which lead to node panic, from recoverable errors. Additionally, the team continued the deployment of a new test network that runs on the Cardano mainnet and computes the stake distribution with the optimized unreleased Cardano CLI command.

Finally, they have designed the certification of generic types of data by a Mithril network and they have started implementing its first phase. Also they fixed a bug that made the client Docker image crash at startup.

Low level overview​

• Worked on the epic that implements eras behavior switch #707:
  • Completed the deployment of the era behavior switch to the test networks #752
  • Completed the enhancement of datum generation for era markers #786
  • Worked on handling the API version with an era switch #727
• Worked on the epic that implements a relational store in the aggregator #779:
  • Worked on the migration/adaptation of the stake_pool table #787
• Completed the refactoring of errors in the signer and aggregator state machines #665
• Worked on testing Mithril with a mainnet Cardano network #777
• Worked on fixing bugs with the Docker client image:
  • Fixed a bug that made the Docker client container crash at startup #769
  • Fixed a bug that prevented the restoration of a snapshot with a Docker client #791
  • Worked on testing the Docker client in the Mithril Client multi-platform test workflow #794

High level summary​

The Consensus team continued working on refactoring and improving the UTxO-HD prototype, and introducing improvements to the lmdb related packages. In particular we identified an opportunity to gain performance by handling locks in a more optimal way.

On the Genesis front, we sketched a mitigation for an issue that PNSol and Researchers caught. We also came op with a road map for not only testing the Genesis prototypes, but also for enriching the tests we already have.

Regarding technical debt, next to some minor improvements, we created component-level micro-benchmarks for adding transactions to the mempool. The results of these benchmarks will be published in the ouroboros-consensus web page.

We also finished moving the Consensus documentation to the ouroboros-consensus repository, released ouroboros-consensus 0.3.0.0, and reduced the time GitHub actions take in ouroboros-network.

Workstreams​

UTxO HD Prototype​

We continued working on refactoring and improving the UTxO-HD prototype. As a result of the first round of sytem-level benchmarks, we identified an opportunity to optimise the way we handle locks to improve performance (#4393).

Also, we introduced several improvements to the lmdb related packages:

• New tests for read-only transaction handles.
• New functions and data types.
• Updates to support improved type safety and smaller constraints in the UTxO HD prototype.
• Windows and MacOS CI support.

Genesis​

We sketched out a mitigation of the issue that PNSol and Researchers caught in the Genesis design.

We came up with a road map for testing the Genesis prototypes, including early milestones that are applicable to today's master branch, ie tests that are useful before Genesis, and that will be nicely enriched when we do add Genesis.

We developed the aforementioned tests, specifically a QuickCheck generator for the Honest leader schedule and one as-aggressive-as-possible Adversarial leader schedule that together satisfy the Praos properties that the Consensus design takes as invariants.

We investigated why the improved ChainDB queueing implementation behaves differently in the baseline compared to the prototype, and we are close to having a full picture of how the Consensus components interact during bulk sync.

Technical debt​

We created component-level micro-benchmarks for adding transactions to the mempool. We plan on extending this to more mempool actions and different types of blocks. We store the benchmark data to make it available to the GitHub action that publishes the benchmarks results.

Other minor improvements include:

• Removal of Test.Util.Classify in favour of Test.StateMachine.Labelling.
• Addition of -Wunused-packages to the default ghc-options for Consensus packages.

Fostering collaboration​

We finished moving the Consensus documentation from ouroboros-network to ouroboros-consensus, in preparation for migrating the code to the latter repository.

Support​

We released ouroboros-consensus 0.3.0.0.

We reduced the load in the ouroboros-network GitHub actions, thereby reducing the time CI jobs take.

2023-03-08 - 2023-03-21

High level summary​

General bug fixes and some refactoring

Completed​

docs​

CI & project maintenance​

Developer experience​

cardano-cli​

• Remove error calls in Cardano.CLI.Shelley.Run.Transaction
• Detect invalid counter and certificate

cardano-api​

• Add missing imports
• Add LedgerStateBabbage and LedgerStateConway pattern synonyms
• Remove unnecessary functions
• Remove duplicate scripts when building transaction body for Mary, Alonzo and Babbage
• Reduce number of calls to toLedgerPParams
• Remove simple script distinction
• Add ToJSON/FromJSON instances for ChainPoint
• Derive Eq instance for AcquiringFailure
• Export fromShelleyBasedScript from Cardano.Api
• Remove unnecessary validation of Data
• Remove hint of "string" support for ScriptData

cardano-node​

• Conway hard forks on prot-ver 9
• Reinstate -Werror for cardano-node and fix errors
• Take all dependencies from Hackage or CHaP
• Less verbose node-to-client and node-to-node version logging
• Get AcceptedConnectionsLimit from the configuration

cardano-testnet​

• fix cardano-testnet Babbage genesis.json

• Release benchmarking: We again performed benchmarks for the next 1.35.6 release candidate.
• New tracing: Backwards compatibility with legacy tracer nomenclature is being implemented to smoothe the transition for end users.
• Analysis pipeline: A major refinement of benchmarking metrics has been realized, along with a structural improvementents regarding metrics denomination.
• Open Sourcing: Work on going live with our benchmarking data has begun, as well as creating an API demo and documentation.
• Nomad backend: The backend was adapted to a major refactoring in workbench and is being equipped with a nomad-exec based task driver.

Performance

1.35.6 release​

Benchmarking the second release candidate for 1.35.6 could again attest to a perfectly clean bill of health.

Analysis pipeline​

Our analysis pipeline has seen an introduction of additional metrics, especially when focusing on the block producing node. They allow us to better differentiate the timing of ledger ticking and mempool snapshotting in the forging loop - a feature that promises much deeper insight into UTxO-HD performance. Additionally, a restructuring of metrics names has been undertaken along with improvements in their data dictionary; a measure that will make benchmarking data more easily accessible.

Open Sourcing​

As a prerequisite for going live with our benchmarking data, we're currently working on consolidation of existing analyses, such as to provide a common foundation when accessing them externally. Additionally, we've begun working on a small visualization demo and interactive API documentation. Those will enable third parties to make use of that data much more easily, by having reliable guidelines and a working example.

Tracing

The new tracing system is being outfitted with a comprehensive mapping of its structure to the legacy tracer nomenclature. This feature will make the switch to the new system as smooth as possible for end users, allowing them to gradually adapt their tooling without breaking any functionality in the process.

Infrastructure

Nomad backend​

The Nomad backend was adapted to the latest major refactoring in workbench. Work was done on making stateful Nomad clients more autonomous, which will greatly facilitate any automation building on that backend. A task driver based on nomad-exec is currently being implemented.