High level summary​

• Benchmarking: Benchmarking node version 8.2.1 has concluded. Additionally we're developing benchmarking setups for GHC9.6 and UTxO-HD.
• Infrastructure: Our analysis pipeline has received improvements reducing memory footprint.
• Tracing: Another batch of optimizations for the new tracing system has been merged; work on namespace consistency guarantees is ongoing.
• Nomad backend: We're performing and analysing various runs for validation purposes on the new hardware cluster.

Low level overview​

Benchmarking​

We've performed and analysed the benchmarks for the 8.2.1 version of cardano-node as part of our release benchmarking cycle.

Setting up cluster benchmarks requires completing full system integration. This applies to both supporting a new build platform, as is GHC9.6, as well as targeting a specific feature, like a UTxO-HD enabled node. Currently, we're working on respective integrations on both those paths.

Infrastructure​

As cluster runs increase in duration, more and more data is accumulated for analysis. Batch analysis mode needs all data to be held in memory, which wouldn't fit anymore even on a 64GB RAM machine. Changes to the in-memory data representation improving on compactness were able to reduce the RAM requirements of our analysis pipeline.

Tracing​

The next portion of optimizations has been completed and merged to master, getting rid of Haskell's native String representation on critical code paths. This concludes the optimization phase of the new tracing system for all its components used by cardano-node.

The implementation for validation of consistent naming and configuration is ongoing. We're splitting out everything that's verifiable at compile time into a seperate test case which we hope to integrate into CI - leaving only configuration constraints to be verified at or before node startup.

Nomad backend​

The verification phase of the nomad cloud backend is ongoing. We're able to perform full runs on the new hardware cluster and porting profiles and configurations from the legacy one. The goal is to reproduce with confidence known regressions, or improvements, between runs performed on the legacy cluster and runs performed with the new backend.

High level summary​

On the UTxO-HD front we reached another important milestone, where we handed over the implementation for benchmarking and testing. We are working on addressing the first issues that were found during the integration tests. On the Genesis front we are finalizing the argument that a checkpoint is not necessary for the initial release, including supporting to support said argument. We are also working on the new node release, and several improvements to our code-base.

UTxO-HD​

• We addressed the issues that came from the benchmarking and testing phase (see #314).
  • We fixed the operational-certificate update exception (#305).

Tech debt​

• We elaborated an inventory of the symbols used by downstream clients in an effort to clearly define the Consensus API, which will help managing changes and their impact to the downstream clients (see this branch).
• We extracted the SOP modules to a separate package as part of our effort to reduce the Consensus API surface area (see this branch).
• We took the first step towards improving how the Cardano hard-fork block is instantiated when running a node (see #275).
• We decreased the flakiness for our test suite (#284).
• We incorporated and merged the pull-request on query versioning (#273).
• We started working on new performance-based regression tests for mempool operations.

Support​

• Nick Frisby continues with his engineer responsibilities. Notably getting the Consensus release out for node pre-release 8.3. Nick also began integrating it into downstream branches (cardano{-api,-cli,-node}). The Node team has now taken over, since Nick had progressed up to the point where they needed to start making new design decisions. Related PRs:
  • #283
  • #301
  • #304
  • #308

The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

The team is currently formalising mini protocols and also further developing the performance modelling prototype.

Details​

• Development of an automated prover for showing conformance of programs to state machines (as part of the mini-protocol framework)

• Completion of the documentation of the (current state of the) mini-protocol framework

• Discussion about two DeltaQ lectures with two practical sessions as part of 4th year masters course on distributed systemsby colleagues at UC Louvain in the Autumn

• Discussion about four DeltaQ lectures by colleagues at U. Bergen as part of a 5th year Masters course in the Autumn

• Adapted original DeltaQ implementation to the new typeclasses and wrote property tests for its algebraic laws

High-level summary​

This week, the Hydra team released version 0.12.0, a significant update that brings support for cardano-node 8.1.2, along with performance enhancements and various API improvements. Additionally, the team successfully onboarded a new contributor who is now actively participating in operating a Hydra Head. They took part in a meeting of the Intersect open source committee and prepared the Hydra master-class for RareEvo, an educational initiative for the community.

If you happen to be at RareEvo, register for the Hydra master-class using this form. We hope to make it possible to follow along on-line as well and will announce details on the Hydra announcements channel on the IOG discord server.

What did the team achieve this week​

• Released version 0.12.0 which adds support for cardano-node 8.1.2, performance and several API improvements. Release notes
• Replaced commit via websocket with external draft + submit. #954
• Onboarded new contributor to the point where he actively participates in operating a Head now.
• Joined a first meeting of the Intersect open source committee
• Prepared the Hydra master-class for RareEvo.

What are the goals of next week​

• Monthly review meeting
• Run the Hydra master-class at RareEvo!
• Updated Hydra tutorial including Mithril and re-record demo/tutorial stuff
• Clear out red bin items (technical debt)
• Groom network resilience and explorer features

High level summary​

The ledger team made great progress in advancing Conway era forward. In particular, voting for DRep and Stake Pool Operators is now at the state that can be used by downstream components. Addition of new ledger state queries makes it possible to verify changes to the governance state, such as placement of votes, submission of governance proposals, inspecting DRep stake distribution and more. Addition of treasury withdrawals concludes the ability to enact all of the governance actions, except for the hard fork initiation.

Low level summary​

Conway progress​

• pull-3577 - Move PParams to GovState
• pull-3609 - Adjust thresholds for sanchonet
• pull-3604 - README: update CIP-1694 link
• pull-3616 - Switch TreasuryWithdrawals to use RewardAcnt
• pull-3599 - Add governance action well-formedness check
• pull-3607 - Rename governance to gov
• pull-3601 - Added DRep anchors
• pull-3615 - Add optional previous GovActionId in some ProposalProcedures
• pull-3596 - Upgradeable families
• pull-3625 - Change constitution to use Anchor
• pull-3592 - Added pulsing incremental computation of the DRep distribution
• pull-3523 - Treasury donations
• pull-3633 - Prevent state changes on phase2 failure
• pull-3634 - Disable serialization of protocolVersion param update
• pull-3630 - Implemented DRep refreshing
• pull-3640 - Remove crypto parameterization from AnchorData
• pull-3636 - Implement DRep ratification with an "always passing"; threshold
• pull-3648 - Governance queries
• pull-3650 - Change environment for ShelleyPOOLREAP rule
• pull-3658 - Set DRep ratify threshold to 51%
• pull-3628 - DELEG, POOL, GOVCERT conformance with Spec v0.8

Testing​

• pull-3619 - Disabled a broken test

Improvements and releasing​

• pull-3660 - Fix bounds and versions
• pull-3661 - Adjust versions, bounds and CHANGELOGs to account for the latest release