High-level summary​

In last sprint we got a performance report of P2P performance testing cluster (which consists of 50 nodes). There is a performance regression in the header notification metric. The P2P cluster is constructed with the same topology as the non-p2p reference one this indicates some regression which needs to be further investigated. This poses a risk for releasing P2P.

We also continued to work on peer sharing: pull #4019.

We continued working on dynamic block production which is required for P2P release for BP nodes: pull #3159.

We simplified the P2P topology format: issue #4559, pull #3888.

We added a new trace point for asynchronous demotions of local peers with Warning severity. This trace is important for SPOs.

Detail description​

Performance regression​

Below we include a graph which shows the performance regression of the P2P code base vs non P2P.

On the x axis is time in seconds which measures the delay from the start of the slot to when a header was received. The y axis is the percentile of nodes that received a header. We are currently investigating possible causes of the regression.

New P2P topology form​

The new topology file format is described in this issue #4559.

Tracing improvements​

• We improved a handshake error reporting, pull #4136
• We added TraceDemoteLocalAsynchronous rendered as DemoteLocalAsynchronous in json format, pull #4127. Such demotions should be investigated by the pool operator. They can indicate a problem in the deployed system, but also they could indicate a remote problem in arranged connections with other SPOs.

Open Source Improvements​

We improved documentation of io-sim and typed-protocols for open-source contributors and/or maintenance tasks: pull #22, pull #45, pull #48.

High level summary​

This week, the hydra team first re-deployed the latest Hydra scripts to the re-spun preview network, see 0.8.0 release notes. They also completed implementation of ADR18 and worked on the validators, but development got impacted by some CI flakyness. The team also met to discuss hard forks & protocol parameter updates #195 and alignment of the specification document with auditors.

What did the team achieve this week​

• Complete and merge ADR18 #579
• Re-deploy hydra scripts to respun preview network, see 0.8.0 release notes #595
• Have first gap of #452 in review.
• Non-achievement: Flaky CI for TUI was impacting us, so we investigated this a lot.
• Engineering meeting to discuss hard forks and protocol parameter updates #195
• Met the internal audit team on the specification to set scope, expectations and collected requirements/open questions.
• Drafted project scope for an external audit RFP.

What are the goals of next week​

• Implement event-sourced persistence #580
• Answer the internal auditor’s questions
• Have a draft RFP ready for a first review internally
• Close some gaps #452

This sprint, the team has been working on the new continuous integration and delivery (CI/CD) pipelines and the automated deployment of environments as part of the new version of the release process. They also coordinated the migration of the pioneer SPO nodes to these new Mithril networks. They have been implementing the automatic data storage upgrade of the signer and the aggregator nodes. Finally, on the crypto side of things, we've implemented an efficiency improvement on the size of the mithril certificates.

Low level overview

• We have been moving forward on the implementation of the release process #500:
  • Setup of the new hosted environments for testing-preview, pre-release-preview and release-preprod with their terraform and GitHub environments #542
  • Adapted the CI workflows to work with the new release process #543
  • Publication of an ADR3
  • Publication of a dev blog post about Mithril networks evolution
  • Releasing our first Mithril distribution 2244.0
• Worked on the API versioning mechanism #565
• Worked on the implementation of the stores migration process for the signer and aggregator nodes #562
• Prepared a Mithril devnet video demo #526
• Implemented a batch Merkle Tree proof, which reduces the size of certificates considerably #484

High Level Summary​

• We've been working toward publishing Cardano Backlog, currently its in review by the IOG communication team.
• We identified a number of libraries which can be published.
• We setup and enhanced cardano-updates.

Detailed description​

I am glad to announce that I was given the role of open-source advocate for cardano project. In last few weeks we were making steps towards publishing our backlog. It's currently under review by the communication team, although most of the issues are already visible across various repositories.

The open-source initiatives have their own project. It is set up to help us track our major open-source activities. Right now there are two work streams:

• the work on Cardano Engineering Handbook;
• libraries to be release on Hackage or crates.io.

We identifies a number of libraries across all the teams which contribute to Cardano which we would like publish to publish, see the following link. Arnauld Bailly recently published quickcheck-dynamic library on Hackage. The networking team is slowly progressing towards publishing io-sim and related packages, checkout the progress here.

Thanks to Arnaud Bailly our Cardano Updates website has a new look & feel! It's using docusaurus.io.

Christian Taylor carried recently a detailed analysis of our open-source repositories. He collected many interesting metrics, which allows us to see where we need to improve as an open-source project to make the Cardano project and many smaller related libraries which we maintain be more open and available for open-source contributors.

The graph below shows which documents the 55 most important Cardano repositories are missing the most: You can expect we will improve in these metrics in the coming weeks.

High level summary​

We made good progress on most of the Action Items we agreed on Lisbon, like:

• Cardano System Tests was fully open to public (tools, tests, results) See cardano-node-tests webpage
• We defined an user-facing-functionality template that is used with the cardano-cli team
  • this includes acceptance criteria & user stories, and definition of done
• We are in the process of running the cardano-node-tests at commit & PR level in cardano-node (we are affected by the Cicero migration right now but we did most of the work already)
• We started to apply a labelling convention on cardano-node issues that will be used to generate some visual dashboards with some metrics [TBD]
• Ziyand Liu started an End-to-End Development and Testing Process for Plutus Features