High-level summary​

This week, the Hydra team released version 0.13.0, which includes fixes and extensions for state persistency and the ability to draft a commit transaction using inline datums.

The team also delivered the first version of the network resilience layer, significantly increasing head availability.

Additionally, they have been providing support to several funded Catalyst projects that want to build on Hydra.

Last but not least, they updated and published security policy and vulnerability disclosure policy reports on how to handle security vulnerabilities within Hydra.

What did the team achieve this week​

• Release 0.13.0
• Published vulnerability reports #1088
• Merged network resilience work part I #1074
• Planning for Cardano Summit participation
• Discussions with funded Catalyst projects wanting to build on Hydra for support
• Merged typos fix PR from @omahs #1095

What are the goals of next week​

• Complete Aiken commit validator script #1072
• Complete Kupo integration #1078
• Brick upgrade on TUI #1103
• Clean backlog
• Prepare and rehearse demo and talk for Cardano Summit

High level summary​

• Benchmarking: Continued benchmarking of UTxO-HD and performed benchmarks for the new tracing system.
• Consensus QTAs: Our protoype approach is applied to potential regression fixes with GHC 9.2.7.
• Development: We've developed strategies for future benchmarks of PlutusV3 and UTxO-HD's on-disk backing store.
• Tracing: The machine-readable tracer configuration has been merged. Optimization of cardano-tracer started.
• Nomad backend: Ongoing variance analysis and refined cluster topology.

Low level overview​

Benchmarking​

Performing and analyzing benchmarks for the UTxO-HD feature is an ongoing effort; we can reliably assess the performance of the in-memory backing store and evaluate possible optimizations (or regressions) for it.

Furthermore, benchmarks of our new tracing system after several rounds of optimization have been performed. The results show all key metrics now being unaffected by the choice of tracing system (legacy or new) - with the new system being able to provide more features and flexibility in comparison. The benchmarks also highlighted further points for optimization, with the focus now on the cardano-tracer service.

Consensus QTAs​

The Quantitative Timeliness Agreements (QTA) prototype is being used in coordination with Consensus and DevX to validate a series of patches that address optmization opportunities which GHC8.10 seizes, but GHC9.2 misses. The feedback from this approach is much more immediate than running benchmarks at system integration level. But once we eventually do, we expect to reproduce the relevant observations - which would mean a big step towards maturing the prototype.

Development​

Benchmarking UTxO-HD's on-disk backing store needs special attention: in virtualized environments, disk I/O is not a reliable metric as it passes several layers of indirection. As this is the very metric which will influence overall performance of this UTxO-HD flavour, we developed a plan to monitor such nodes, connected to a running network, on dedicated hardware - having direct SSD access. Replicating this setup for an entire benchmarking cluster of such nodes will be a future effort.

PlutusV3 will come with new builtins and a new cost model. It will take a specialized benchmark to ascertain the soundness of that model running a full cluster of nodes, possibly stressing expensive builtins. At the same time, we'd like to validate the many improvements that have gone into the Plutus evaluator.

Tracing​

The focus for further optimization of the new tracing system has shifted to cardano-tracer - the service receiving and processing traces from one (or more) nodes. Whilst undisputed that the code living in cardano-node is more performance critical, the receiving service must still minimize its resource footprint. Moreover, it can generate load for a running node when querying data points from it - which calls for tight control of that mechanism and its possible configurations.

Nomad backend​

Variance analysis of new nomad backend has revealed a necessary adjustment of the cluster's topology. We repeated the same analysis and now see even better confidence in the measurements taken with nomad. This concludes the work on the backend proper for the time being. The last steps before production use will focus on the interface between backend and our workbench, which provides all high-level benchmark definitions and analysis machinery.

High level summary​

The value-only workload benchmarks showed that the mempool forging regression observed in the UTxO-HD branch was fixed by the latest patch. In spite of the higher resource demands, for the metrics of interest (forging, peer-propagation, end-to-end propagation) we see no regression when using the UTxO-HD version of Cardano node, with the in-memory backend.

On the Genesis front the Researchers continue reviewing different aspects of the design, in particular the argument that the Genesis rule will select the Cardano historical chain. We also merged a fix for the Babbage to Conway transition, and released a new version of Consensus.

Genesis​

• We elicited review from the Researchers on a final draft of the argument that the Genesis rule will select the Cardano historical chain (392).

Support​

• We merged a minimal patch that fixes parameter update bug during the Babbage to Conway transition (366).
• We enabled richer tracers in cardano-node that can be useful in future debugging (384).
• Esgen continues with his release engineer activities, and created a new Consensus release.

Fostering collaboration​

• We merged a new section into our documentation that explains the existing hard-fork combinator (HFC) interface and its complexities, which are relate do why the Babagge to Conway transition surprised us in this way. This explanation is step one towards improving the HFC interface (369).

High level overview​

This week, the Mithril team updated a proposal for implementing Mithril in a light wallet and created a discussion on implementing Mithril in a light wallet to receive feedback and contributions from the community.

They also completed a proof of concept to run a Mithril client in a browser with WASM and worked on enhancing the explorer UI with SPO tickers displayed next to the pool IDs. Additionally, the team finalized a tool for benchmarking aggregator performance over a range of signer and client loads and for visualizing the results of these benchmarks.

Finally, the team added a root page to the aggregator endpoint, created a status page for the Mithril networks, and added a section about KES key rotation in the signer documentation for SPOs.

Low level overview​

• Created the discussion Implement Mithril in a light wallet #1273
• Completed the issue Run client in browser WASM PoC #1254
• Completed the issue Benchmark aggregator performances #1220
• Worked on the issue Add SPO tickers in explorer #1185
• Completed the issue Add a non 404 status code on the aggregator endpoint #1103
• Completed the issue Use source attribute in errors #1265
• Completed the issue Create a SPO checklist for KES keys update #1267
• Completed the issue Configure status page and alerting #1277
• Completed the issue Upgrade dependencies #1274

High-level overview of sprint 45​

Bootstrap Peers​

We started reviewing the bootstrap peers PR, ouroboros-network#4615.

Towards Typed Protocols 0.2.0.0​

We discovered a performance regression when using typed-protocols-0.2.0.0, and we started investigating where it comes from. Currently, we see that typed-protocols-0.2.0.0 can outperform typed-protocols-0.1.0.0 when running in isolation with a simple ping-pong protocol, so the regression might be in the new block fetch implementation which comes with typed-protocols-0.2.0.0 See typed-protocols#3.

Tech Debt​

We merged two PRs written by Galois engineers:

• a pull request which refactors the main entry function for P2P, see ouroboros-network#3834;
• a pull request which reviews usage of unsafe function in the network code based.

Galois also made progress with the following two issues:

• add microbenchmarks to some of our tests: ouroboros-network#3832
• inspect use of error in ouroboros-network repository: ouroboros-network#3836

IO-Sim​

IOSimPOR​

We found and fixed a bug in IOSimPOR. We'd like to thank Prof. John Hughes (Quviq AB) for helping us with debugging the issue.

We also provided a more uniform API for IOSimPOR, and added ways to make the debugging similar problems in the future easier.

IOSim​

To prepare for the next release, we consolidate packages taking advantage of the public sublibraries supported now both by cabal and Hackage. This is a work in progress, io-sim#114.

Cardano Newtork Service Assurance​

Galois made the following progress:

• A test run of spinning up a CNSA instance was done, as a result documentation was updated.
• Based on the IOG code review of the CNSA code, updates to the CNSA code were made.
• Galois has started the design for adding a CNSA analysis for "fetched bytes over time while node is syncing".

P2P adoption​

In the last two weeks, we've seen increase in P2P adoption.

The following graphs show several different versions of relays running on the mainnet. The green line NodeToNodeVersionV10.True denotes P2P relays.

Open Source​

We upstreamed our FFI bindings to Windows named pipes to Win32 package, the PR was accepted and merged.

We also received an external contribution which enhanced our documentation, see ouroboros-network#4676.