Skip to main content

· 2 min read
Sebastian Nagel

High-level summary​

This week, the Hydra team released version 0.8.1, which includes several fixes and a user-wished extension of the persistence introduced by 0.8.0 of replaying server outputs to make clients like the hydra-tui be aware of the latest hydra-node state. The team also worked on the specification and closing gaps in the on-chain scripts, collaborated with the education team on a Hydra tutorial and also renamed the repository from hydra-poc to hydra!

What did the team achieve this week​

  • Implemented replaying of server outputs to address #580
  • Released version 0.8.1 containing this and other fixes Release notes
  • Worked on the bounded tx validity as one of the on-chain script fixes, but couldn’t finish it just yet #615
  • Collaborated with the education team on a Hydra tutorial.
  • Discovered and discussed issues with the “seen ledger”.
  • Renamed the Hydra repository hydra-poc -> hydra

What are the goals of next week​

  • Create and discuss an ADR about handling tx validity correctly.
  • Attend the Cardano Summit in Lausanne. Hydra will be topic of one talk! Save the date+time: "Developing Hydra" on Day 2, Nov 21st, 13:50 CET
  • Have a team workshop / hackathon after the summit:
    • Retrospective
    • Roadmapping session
    • Hack on something complex or useful

· 2 min read
Iñigo Querejeta Azurmendi

High level overview​

The Mithril team has released their second distribution 2246.1 following the activation of the alpha version of the new release process. They continued refining its implementation and added a new set of artifacts to the distributions, such as Debian packages and macOS/Windows binaries. The team also enhanced the Mithril Explorer, which now provides more detailed information about epoch settings and easier access to multiple aggregators for the users. They have also worked on an enhanced mechanism for node versioning, storage, and communication protocol. They also implemented version detections for the signer and aggregator nodes, designed the automatic store upgrade feature for these nodes, and enhanced the documentation of the configuration parameters of the several Mithril networks.

Finally, the team continued working on the elaboration of the CIP that will allow the decentralization of Mithril by relying on the Cardano node network layer.

Low level overview​

  • Released a new Mithril distribution 2246.1
  • Completed the first stage of the store migrations process #562
  • Added a Mithril API version that is now exposed in the headers of the requests sent and received by the nodes #565
  • Enhanced the explorer UI with epoch settings information and aggregators management on browser local storage #576
  • Prepared a Daedalus synchronization benchmark video with/without Mithril #606
  • Upgraded the Cardano nodes of the testing Mithril networks to 1.35.4 #594
  • Worked on implementing SPO tests nodes on testing Mithril networks #563
  • Worked on the CIP design for Mithril piggybacked on the Cardano network layer #588
  • Worked on the refactorizaton of the aggregator multi signer engine #398

· 6 min read
Damian Nadales

High-level summary​

During the past two weeks, the consensus team started documenting the implementation of the UTxO HD feature and continued developing tests for it. As part of our work on UTxO HD, we improved the Haskell support for LMDB. We also spent time working on the LSM tree prototype, and designed a parameter tuning algorithm for it. Regarding our work on Genesis, our investigation of the "plateaus" pointed at the TICKF slowdown on era boundaries as culprit. This led us to developing a caching strategy that will not only remove the aforementioned "plateaus", but can help alleviating the growing block production delay on epoch switch. We also helped reviewing the block forge credential hotswap feature, which is intended for use in the adoption of P2P.

We also worked on paying technical debt and fostering collaboration. In particular, we improved the io-sim framework, which is crucial for testing and simulating Cardano components. We also removed thunks that appeared on era translations, and improved our diffusion pipelining feature. We are working on a presentation for explaining Praos and Genesis.

High-level status report​

  • Finish the UTxO HD prototype: in progress.
    • We added documentation for this feature.
    • We developed the second version of the mempool tests.
    • We fixed benchmarks that were inflating the speedup we observed in the anti-diff implementation of sequences of differences. Speedups are now in the range of [3.33, 4.75], which remain significant.
    • We continued improving Haskell LMDB support.
    • We finished implementing a "parameter tuning algorithm" for the LSM tree prototype. This enables us to run experiments to check the correctness of the algorithm.
  • Genesis: in progress.
    • Work investigating the "plateaus" in the ChainSync jumping prototype pointed to the TICKF slowdown on era boundaries as culprit.
  • Tech debt:
    • We improved the capabilities of our io-sim library, which is crucial for testing and simulating Cardano components.
    • We removed thunks from epoch translations in the ledger.
    • We added Linux CI support for lmdb-simple.
    • We got pending diffusion pipelining improvements merged.
  • Fostering collaboration:
    • We are working on a explanation of Praos and Genesis protocols.
  • Support:
    • Investigation of CSJ "plateaus" led us to developing a caching strategy for TICKF that will not only remove these "plateaus", but can help alleviating the growing block production delay on epoch switch.
    • We reviewed the block forge credential hotswapping feature which is intended for use in the adoption of P2P.

Workstreams​

Finish the UTxO HD prototype​

We merged PR #4060, which adds a report documenting the UTxO HD feature, and puts emphasis in explaining how the mempool works in combination with UTxO HD.

We opened a draft PR with the second iteration of the property tests for the mempool (#4076).

We fixed the Arbitrary instances for keys and values in DiffSeq benchmarks (#4143). The problem was that we were testing with mostly small values, which artificially boosted the performance gains we saw on benhcmarks. Speedups are now in the range of [3.33, 4.75] across the different configurations.

Backing store property tests​

We focused on incorporating feedback on the monadic cursor API PR (#1). This required us to make small tweaks to quickcheck-lockstep to test the new API. We also updated the backing store property tests to use the new version of the monadic cursor API.

LSM tree implementation​

We worked on the LSM tree prototype. In particular: finished implementing a "parameter tuning algorithm" that adapts the LSM tree design based on factors like:

  • workload
  • machine specs,
  • and characteristics of the data being stored.

We are now running experiments to gather results and cross-reference them with existing experimental results from the LSM tree paper to see if the algorithm is working correctly.

Benchmarking the CSJ prototype​

We focused on investigating the "plateaus" in the ChainSync tip, which turned out to be due to the TICKF bug which we previously were only aware of in the context of the long forging times near epoch boundaries. For the most drastic patch by @nfrisby to speed up TICKF, full sync is speeding up by 7%.

The following plot shows that by caching the TICKF the ChainSync tip and the VolatileDB tip progress at the same rate.

The plot below shows the speedup observed by caching the TICKF rule wrt the baseline.

Technical debt​

After addressing the PR comments, we merged PR #16, which implements the MonadCatch instance for STM. This extends the capability of our io-sim library, which is crucial for testing and simulating Cardano components PR #16 closed #1461. This new feature was published as version 0.4.0.0 of io-sim.

We continued with our work fixing the NoThunk errors required for enabling nightly tests, with the help of TVarInvariant checks in strict-stm and nothunks libraries. We proposed fixes in cardano-ledger that took care of thunks that appeared in era translations (#3143). The fixes will be integrated back into consensus when cardano-ledger approves and publish the changes introduced in #3143.

We added CI support for lmdb-simple (#2). We currently test the build on a Linux environment only.

We got pending diffusion pipelining PRs (#3857, #3860, #3856) merged, after rebasing and addressing feedback.

Fostering collaboration​

@nfrisby finished a visualisation tool and outlined scripts for the Praos and Genesis explanation presentations. The idea is to produce a video that gives an overview of these protocols.

Support​

We started working on caching the computation of the TICKF rule (#4054), since this was blocking our benchmarking work for Genesis. In addition, this issue has the Cardano community quite concerned, so we are hoping the work done in caching the computation of the TICKF rule can help alleviating the growing block production delay on epoch switch.

We reviewed the block forge credential hotswapping PR #3800 from the networking team, which is intended for use in the adoption of P2P.

· One min read
Jordan Millar

2022-11-16 - 2022-11-29

High level summary​

Completed​

docs​

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

In Progress​

cardano-cli​

cardano-api​

cardano-node​

· 2 min read
Serge Kosyrev

High level summary​

  1. P2P performance investigation is ongoing, in support of the networking team.
  2. SECP benchmarking enablement is underway: we already have the script and are working on Plutus V2 generation support.
  3. Unexpected setback in the new tracing system: full scale benchmarks have shown a performance regression: local chain syncing benchmarks were an improvement over legacy tracing.
  4. On the open sourcing front we added an integrated data dictionary, which is necessary for explaining ourselves to the world. SRE collaboration on production deployment of performance data publishing has started.
  5. We have started bringing the Nomad-based workbench backend closer to a cloud deployment scenario.

Performance​

We are supporting the networking team on P2P performance investigation. Generation support for Plutus V2 was started. We have collaborated with the Plutus team to get a SECP benchmark script, which is now ready for use, pending Plutus V2 support. The transaction generator has also been updated to the cardano-api changes.

Tracing​

We ran an initial round of full-scale benchmarks for the new tracing system -- which uncovered a regression relative to legacy tracing, which is contrary to the local chain syncing benchmarks, that showed improvement instead. We added tracing to cardano-tracer, fixing some minor bugs on the way. Network and disk IO metrics are now collected once again and are integrated into analysis.

Infrastructure​

The first iteration of the Nomad-based local workbench backend was completed -- it has reached feature parity with the existing supervisor backend. The next iteration started, bringing it closer to the cloud scenario, by deploying to separate Nomad tasks connected by a virtual network. This will serve as basis for CI and full cloud backends.

We designed and implemented the authoring pipeline for the performance data dictionary, which will be henceforth embedded in our performance reports. We are collaborating with SRE on production deployment of data publishing.

A number of smaller workbench, data analysis & reporting improvements have been made.