The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

This sprint the team has been working on drafts of two papers improving the deltaQ tutorial, consulting on performance design, and the final version of the Djed paper.

Details​

• Preparing draft of DeltaQSD algebra paper for ICE 2023

• Successful outcome of Marlowe consultancy leading to substantial improvement in performance

• Preparing sections on the communication language and bisimulation proofs for draft paper about verifying design refinements for distributed system design

• Camera ready version of Djed paper under preparation

High-level summary​

This week, the Hydra team addressed issues with committing complex UTxOs into a Hydra Head, prepared the hydra-node to be run on mainnet (soon) and improved the test suite of the hydra-plutus scripts. The specification is constantly getting improved upon reviewer feedback on overleaf - this week the graphical notation was refined to be more transaction-focused.

What did the team achieve this week​

• Clarified “bug” of committing UTxOs with reference scripts to a Hydra head #737
• Prepare usage of hydra-node on mainnet by updating command line options #715
• Improved our mutation test suite to all expected errors #705
• Re-opened our persistent hydraw demo head on preprod using version 0.9.0
• Updated on-chain graphs to be more transaction-focused in the specification

What are the goals of next week​

• Complete mainnet compatibility feature
• Prepare the Hydra workshop
• Address all todo’s in the Hydra specification
• Unblock the auctions project with “commit from script” a solution

High level overview​

The Mithril team worked on deploying the era activation mechanism to their test networks and preparing the upgrade procedure to be implemented on the signer nodes run by the SPOs. They also refactored the state machines of the aggregator and signer nodes so that they can better handle critical errors, which lead to node panic, from recoverable errors. Additionally, the team continued the deployment of a new test network that runs on the Cardano mainnet and computes the stake distribution with the optimized unreleased Cardano CLI command.

Finally, they have designed the certification of generic types of data by a Mithril network and they have started implementing its first phase. Also they fixed a bug that made the client Docker image crash at startup.

Low level overview​

• Worked on the epic that implements eras behavior switch #707:
  • Completed the deployment of the era behavior switch to the test networks #752
  • Completed the enhancement of datum generation for era markers #786
  • Worked on handling the API version with an era switch #727
• Worked on the epic that implements a relational store in the aggregator #779:
  • Worked on the migration/adaptation of the stake_pool table #787
• Completed the refactoring of errors in the signer and aggregator state machines #665
• Worked on testing Mithril with a mainnet Cardano network #777
• Worked on fixing bugs with the Docker client image:
  • Fixed a bug that made the Docker client container crash at startup #769
  • Fixed a bug that prevented the restoration of a snapshot with a Docker client #791
  • Worked on testing the Docker client in the Mithril Client multi-platform test workflow #794

High level summary​

The Consensus team continued working on refactoring and improving the UTxO-HD prototype, and introducing improvements to the lmdb related packages. In particular we identified an opportunity to gain performance by handling locks in a more optimal way.

On the Genesis front, we sketched a mitigation for an issue that PNSol and Researchers caught. We also came op with a road map for not only testing the Genesis prototypes, but also for enriching the tests we already have.

Regarding technical debt, next to some minor improvements, we created component-level micro-benchmarks for adding transactions to the mempool. The results of these benchmarks will be published in the ouroboros-consensus web page.

We also finished moving the Consensus documentation to the ouroboros-consensus repository, released ouroboros-consensus 0.3.0.0, and reduced the time GitHub actions take in ouroboros-network.

Workstreams​

UTxO HD Prototype​

We continued working on refactoring and improving the UTxO-HD prototype. As a result of the first round of sytem-level benchmarks, we identified an opportunity to optimise the way we handle locks to improve performance (#4393).

Also, we introduced several improvements to the lmdb related packages:

• New tests for read-only transaction handles.
• New functions and data types.
• Updates to support improved type safety and smaller constraints in the UTxO HD prototype.
• Windows and MacOS CI support.

Genesis​

We sketched out a mitigation of the issue that PNSol and Researchers caught in the Genesis design.

We came up with a road map for testing the Genesis prototypes, including early milestones that are applicable to today's master branch, ie tests that are useful before Genesis, and that will be nicely enriched when we do add Genesis.

We developed the aforementioned tests, specifically a QuickCheck generator for the Honest leader schedule and one as-aggressive-as-possible Adversarial leader schedule that together satisfy the Praos properties that the Consensus design takes as invariants.

We investigated why the improved ChainDB queueing implementation behaves differently in the baseline compared to the prototype, and we are close to having a full picture of how the Consensus components interact during bulk sync.

Technical debt​

We created component-level micro-benchmarks for adding transactions to the mempool. We plan on extending this to more mempool actions and different types of blocks. We store the benchmark data to make it available to the GitHub action that publishes the benchmarks results.

Other minor improvements include:

• Removal of Test.Util.Classify in favour of Test.StateMachine.Labelling.
• Addition of -Wunused-packages to the default ghc-options for Consensus packages.

Fostering collaboration​

We finished moving the Consensus documentation from ouroboros-network to ouroboros-consensus, in preparation for migrating the code to the latter repository.

Support​

We released ouroboros-consensus 0.3.0.0.

We reduced the load in the ouroboros-network GitHub actions, thereby reducing the time CI jobs take.

2023-03-08 - 2023-03-21

High level summary​

General bug fixes and some refactoring

Completed​

docs​

CI & project maintenance​

Developer experience​

cardano-cli​

• Remove error calls in Cardano.CLI.Shelley.Run.Transaction
• Detect invalid counter and certificate

cardano-api​

• Add missing imports
• Add LedgerStateBabbage and LedgerStateConway pattern synonyms
• Remove unnecessary functions
• Remove duplicate scripts when building transaction body for Mary, Alonzo and Babbage
• Reduce number of calls to toLedgerPParams
• Remove simple script distinction
• Add ToJSON/FromJSON instances for ChainPoint
• Derive Eq instance for AcquiringFailure
• Export fromShelleyBasedScript from Cardano.Api
• Remove unnecessary validation of Data
• Remove hint of "string" support for ScriptData

cardano-node​

• Conway hard forks on prot-ver 9
• Reinstate -Werror for cardano-node and fix errors
• Take all dependencies from Hackage or CHaP
• Less verbose node-to-client and node-to-node version logging
• Get AcceptedConnectionsLimit from the configuration

cardano-testnet​

• fix cardano-testnet Babbage genesis.json