High-level summary​

This week, the Hydra team conducted the monthly review meeting for Agust,which continously ensures transparent communication and project evaluation.

In terms of community engagement, the Hydra team gave a Hydra master-class at RareEvo workshop as an educational initiative for the community. Additionally, the team updated the Hydra demo and tutorial to include Mithril and latest release changes.

They also focused on exploring the Hydra network resiliance in order to improve hydra-node availability, as it becomes stuck very easily and that leads to a bad user experience.

What did the team achieve this week​

• Hydra master-class at RareEvo workshop!
• Monthly report & review meeting
• Updated Hydra tutorial including Mithril and re-record demo/tutorial stuff
• Groomed network resilience and explorer features #188
• New ADR for “Resourced based API” #1028

What are the goals of next week​

• Have a poc on network resilience working #188
• Fix issue #931: “Chain state in head state not updated on replayed observation”
• Fix issue #1039: "Close transaction dropped from cardano-node"

High level summary​

• Benchmarking: Benchmarking node version 8.2.1 has concluded. Additionally we're developing benchmarking setups for GHC9.6 and UTxO-HD.
• Infrastructure: Our analysis pipeline has received improvements reducing memory footprint.
• Tracing: Another batch of optimizations for the new tracing system has been merged; work on namespace consistency guarantees is ongoing.
• Nomad backend: We're performing and analysing various runs for validation purposes on the new hardware cluster.

Low level overview​

Benchmarking​

We've performed and analysed the benchmarks for the 8.2.1 version of cardano-node as part of our release benchmarking cycle.

Setting up cluster benchmarks requires completing full system integration. This applies to both supporting a new build platform, as is GHC9.6, as well as targeting a specific feature, like a UTxO-HD enabled node. Currently, we're working on respective integrations on both those paths.

Infrastructure​

As cluster runs increase in duration, more and more data is accumulated for analysis. Batch analysis mode needs all data to be held in memory, which wouldn't fit anymore even on a 64GB RAM machine. Changes to the in-memory data representation improving on compactness were able to reduce the RAM requirements of our analysis pipeline.

Tracing​

The next portion of optimizations has been completed and merged to master, getting rid of Haskell's native String representation on critical code paths. This concludes the optimization phase of the new tracing system for all its components used by cardano-node.

The implementation for validation of consistent naming and configuration is ongoing. We're splitting out everything that's verifiable at compile time into a seperate test case which we hope to integrate into CI - leaving only configuration constraints to be verified at or before node startup.

Nomad backend​

The verification phase of the nomad cloud backend is ongoing. We're able to perform full runs on the new hardware cluster and porting profiles and configurations from the legacy one. The goal is to reproduce with confidence known regressions, or improvements, between runs performed on the legacy cluster and runs performed with the new backend.

High level summary​

On the UTxO-HD front we reached another important milestone, where we handed over the implementation for benchmarking and testing. We are working on addressing the first issues that were found during the integration tests. On the Genesis front we are finalizing the argument that a checkpoint is not necessary for the initial release, including supporting to support said argument. We are also working on the new node release, and several improvements to our code-base.

UTxO-HD​

• We addressed the issues that came from the benchmarking and testing phase (see #314).
  • We fixed the operational-certificate update exception (#305).

Tech debt​

• We elaborated an inventory of the symbols used by downstream clients in an effort to clearly define the Consensus API, which will help managing changes and their impact to the downstream clients (see this branch).
• We extracted the SOP modules to a separate package as part of our effort to reduce the Consensus API surface area (see this branch).
• We took the first step towards improving how the Cardano hard-fork block is instantiated when running a node (see #275).
• We decreased the flakiness for our test suite (#284).
• We incorporated and merged the pull-request on query versioning (#273).
• We started working on new performance-based regression tests for mempool operations.

Support​

• Nick Frisby continues with his engineer responsibilities. Notably getting the Consensus release out for node pre-release 8.3. Nick also began integrating it into downstream branches (cardano{-api,-cli,-node}). The Node team has now taken over, since Nick had progressed up to the point where they needed to start making new design decisions. Related PRs:
  • #283
  • #301
  • #304
  • #308

The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

The team is currently formalising mini protocols and also further developing the performance modelling prototype.

Details​

• Development of an automated prover for showing conformance of programs to state machines (as part of the mini-protocol framework)

• Completion of the documentation of the (current state of the) mini-protocol framework

• Discussion about two DeltaQ lectures with two practical sessions as part of 4th year masters course on distributed systemsby colleagues at UC Louvain in the Autumn

• Discussion about four DeltaQ lectures by colleagues at U. Bergen as part of a 5th year Masters course in the Autumn

• Adapted original DeltaQ implementation to the new typeclasses and wrote property tests for its algebraic laws

High-level summary​

This week, the Hydra team released version 0.12.0, a significant update that brings support for cardano-node 8.1.2, along with performance enhancements and various API improvements. Additionally, the team successfully onboarded a new contributor who is now actively participating in operating a Hydra Head. They took part in a meeting of the Intersect open source committee and prepared the Hydra master-class for RareEvo, an educational initiative for the community.

If you happen to be at RareEvo, register for the Hydra master-class using this form. We hope to make it possible to follow along on-line as well and will announce details on the Hydra announcements channel on the IOG discord server.

What did the team achieve this week​

• Released version 0.12.0 which adds support for cardano-node 8.1.2, performance and several API improvements. Release notes
• Replaced commit via websocket with external draft + submit. #954
• Onboarded new contributor to the point where he actively participates in operating a Head now.
• Joined a first meeting of the Intersect open source committee
• Prepared the Hydra master-class for RareEvo.

What are the goals of next week​

• Monthly review meeting
• Run the Hydra master-class at RareEvo!
• Updated Hydra tutorial including Mithril and re-record demo/tutorial stuff
• Clear out red bin items (technical debt)
• Groom network resilience and explorer features