High-level summary​

This week, the Hydra team has worked on cleaning up several things in progress after last week's summit. They have extended their model-based testing (MBT) approach with transaction creation & observation #410, solved AcquirePointTooOld problems of the hydra-node with by changing the wallet initialization #439. Also, the Hydra researchers updated the security proofs of the Coordinated Hydra Head, which are bound to be included in the Hydra HeadV1 specification.

What did the team achieve this week​

• Monthly review & report - will also be published on our website #644
• Extended the model-based testing (MBT) with transaction creation/observation #410
• Solve AcquirePointTooOld problems with new wallet initialization #439
• Fixed our hydraw deployments (EC2 instances)
• Created & discussed ADR21 within tx validity work
• Received & discussed security proofs of Coordinated Hydra Head (requires more work)

What are the goals of next week​

• Get ADR21 accepted & close tx validity gap in our implementation
• Complete review & integrate the Hydra tutorial
• Review latest hydra-pay work
• Have a LaTeX write-up of the HeadV1 (Coordinated Hydra Head) spec
• Get Cicero (new CI) working

High level summary​

The DB Sync team prepared a release 13.1.0.0-rc2 which includes many improvements for db-sync, it makes rollbacks and syncing much faster, simplifies the schema, fixes bugs and introduces migrations. This release finalises the objectives that were set for db-sync for the previous 3 months period and part of the syncing speed objective set for the next period Changelog

Lower level summary​

• Branch release/13.1.0.x includes all the improvements related to the release. The release is passing through the testing phase and a number of bugs and issues have been fixed, like #1312 #1311. Also many new unit tests have been added.

• Part of the release branch is cherry-picked back into master, in a way that it respects the new release and development process, so that it takes into account migrations release process

• The DB Sync team has also tagged release 13.0.6 which better supports preview and preprod for docker.

High level overview​

The Mithril team worked on finalizing their release process by adding new features: publishing their cryptographic library to the Rust community, adding node versions manifest in the release notes, and signing the binaries embedded in the distributions. They deprecated the declarative signer node registration that will be decommissioned in a few weeks. The team also completed the automatic store upgrade process for the signer and aggregator nodes.

Finally, the team continued working on the redaction of the CIP that will allow the decentralization of Mithril by using the Cardano networking layer.

Low level overview​

• Implemented custom Mithril SPOs on testing/pre-release networks #563
• Deprecated Signer Declarative Pool Id registration mode #585
• Completed the second stage of the store automatic migration process #600
• Completed the deployment pipelines to crates.io registry #588
• Completed automatic generation of nodes/libraries versions manifest in releases notes #599
• Completed CI/CD handling of PR from forks #597
• Worked on the CIP design for Mithril piggybacked on the Cardano network layer #586
• Worked on signing the artifacts released in the distributions by the CI/CD #587
• Worked on multi-platforms end to end test #601
• Worked on the refactorizaton of the aggregator multi-signer engine #398

High level summary​

During the last 2 weeks we did more improvements on our Test Framework, planned the testing of the P2P Single Relay functionality, and also tested some DB-Sync tags.

Workstreams​

Framework improvements:​

• moved the System Test CLI Pipelines from BuildKite to Github Actions
• improved the reporting tools to support the rerun of the failled tests and update of the reports
• added support for Github API in report-aggregator, so reports will be generated from the GitHub nightly jobs from now on
• added support for mixed topology - P2P, legacy, mixed topologies
• planned the P2P Single Relay system test activities
• added support to start regression tests with PV8 + better selection of tests

DB-Sync:​

• confirmed that DB-Sync release 13.0.5 is compatible with Node release 1.35.4 + Protocol Version 8 (on the Preview environment)
• tested a couple db-sync tags - 13.1.0.0-rc1, 13.1.0.0-rc2

High level summary​

During the past two weeks, the consensus team merged improvements to the monadic cursor API that was needed to implement LMDB range reads, which is in turn required for the implementation of the UTxO HD feature. We added tables to several tests in for the UTxO HD feature, which increases our confidence in the correctness of the prototype. The mempool property tests are close to being completed. Also, we finished the LSM tree tuning algorithm.

On the Genesis front we started simplifying the BlockFetch logic with CSJ-specific workloads in mind.

We are also documenting the Block Diffusion Pipelining feature, and added a high-level overview of consensus to the top level documentation of ouroboros-network.

Workstreams​

UTxO HD prototype​

We merged the implementation of a monadic cursor API (#1)) which was needed to solve a bug with LMDB range-reads. After this PR was merged, we focused on bridging the gap between the lmdb-simple interface and consensus by facilitating using lmdb-simple's cursor API without Serialise constraints (#3).

We refactored the backing store property tests to use quickcheck-lockstep (#4081).

We added tables to the mock ledger in the UTxO-HD feature branch (#4184). Every test that used to run with SimpleBlocks now uses tables. This will enable us to exercise the UTxO HD mempool integration by leveraging the existing mempool property-tests. The new state-machine property-tests are still needed for testing the parallel behaviour of the mempool.

Our work on the mempool state-machine tests revealed the need for improvements in the quickcheck-state-machine library. Parallel testing assumed that the state machine did not have access to mutable references. However, the mempool tests require the use of such mutable references for mocking the ledger interface. As a result, our parallel tests were failing with rather obscure messages. @Jasagredo submitted a pull request (#12) that allows for new mutable references to be created at each run of the state machine.

Backing store property tests​

LSM tree implementation​

We finished the LSM Tree tuning algorithm. We are currently tidying up the code and gathering results (i.e., plots and their interpretation).

CSJ prototype​

We started simplifying the BlockFetch logic with CSJ-specific workloads in mind.

New VRF and KES crypto integration​

Started working on supporting new version of StandardCrypto which uses compact KES and batched VRF (#4151).

Technical debt​

We reviewed the existing state of the Block Diffusion Pipelining document. We are now working on the "Implementation" section (#4020).

Fostering collaboration​

We cleared up our understanding of the error dynamics of forecasting (#4146 and #4174).

We submitted a pull request that adds an overview of consensus to the top level documentation of ouroboros-network (#4197). This overview describes the consensus components and adds a hyperlinked map to the modules documentation.

https://github.com/input-output-hk/ouroboros-network/pull/4197