Skip to main content

· 2 min read
Damian Nadales

High level summary​

During the past two weeks the team working on the Genesis implementation continued to engage with the researchers, which resulted in various simplifications of the correctness argument for the historical Genesis window. They also decided on an approach for a syncing node to decide that it is (no longer) caught up. This functionality was requested by the networking team.

The team working on the UTxO-HD implementation ran ad-hoc benchmarks that showed performance issues, which are being investigated. They also merged several improvements required for the first UTxO-HD release, and added a package for easing integration with other downstream components.

Regarding our support activities, we integrated the latest Ledger changes into Consensus in preparation for release 8.2 of node.

Genesis​

  • We continued to engage with the researchers on our probabilistic model for historical Genesis window, resulting in various simplifications that make the correctness argument more clear while not being excessively conservative.

  • We decided on an approach of how to implement functionality requested by the Networking team; namely, how a syncing node can safely conclude that it is (no longer) caught up. Certain parameters are still subject to discussion with the researchers, and we have still have to agree on a concrete API for this functionality with the Networking team.

UTxO-HD​

  • We merged the last of the PRs that were part of UTxO-HD improvements for version 0.1: expose UTxO-HD configuration options in the node, refactor ledger tables, and expose a method of computing the UTxO set size.
  • We added a new "legacy" cardano block in a new ouroboros-consensus-cardano-legacy-block package that should ease the transition for some downstream packages to UTxO-HD, like db-sync. This is really only useful for downstream packages that use the parts of consensus that don't involve the storage components, in which case we can largely ignore ledger tables. Ignoring ledger tables could also make functionality like block (re-)application more performant for the legacy Cardano block as compared to the actual (UTxO-HD compatible) Cardano block.
  • We performed ad-hoc benchmarks of the UTxO-HD implementation, observing a regression in sync speed in the LMDB implementation as well as a regression in memory usage on the in-memory implementation. We are investigating this.

· 2 min read
Michael Karg

High level summary​

  • Benchmarking: We've performed several new benchmarks and a performance investigation in preparation of switching the default compiler to GHC9.
  • Infrastructure: The first batch of refactoring and documentation for our tx-generator has been merged to master.
  • Tracing: We've looked into an issue where the tracing system's concurrency could prevent a graceful node shutdown.
  • Nomad backend: Our new cloud backend has seen various improvements regarding deployment and monitoring; validation runs for the backend are ongoing.

Low level overview​

Benchmarking​

The compiler switch to GHC9 as the default build platform for cardano-node and its components still has noticeable effects on system-wide performance metrics. An investigation into the different resource usage profiles of compiler versions does seem to indicate GHC9's significantly different inlining behaviour may produce those effects. We're currently locating the specific places in component code that have the most extensive effect in that regard.

Using the forge-stress approximation we set up, we could determine that above effect is not due to a range of RTS parameters, as for example the number of capabilites used by the node.

Infrastructure​

The tx-generator is a crucial part of our tooling responsible for producing very specific workloads for our benchmarking cluster. In an effort to flesh out an API to make it reusable for more general use cases, a first set of refactorings has been merged to master. Additionally, this merge contained systematic documentation both for internal and for exposed areas of the code base.

Tracing​

The tracing system's concurrency could under certain conditions prevent a graceful shutdown of the node. This issue did occur only after adding specific new traces on a development branch. We could localize and address that issue.

Nomad backend​

With the data gathered from running the new nomad cloud backend, we've been able to address many, many small and medium-sized improvements. The deployment process has been restructured for better efficiency, and the healthcheck system could be fine-tuned to recognize severity of various conditions that might occur. Optimization of fetching all run data from the cloud for evaluation is in progress.

Additionally, we're continuing the new backend's validation by setting up test runs and looking into comparative analyses with metrics gathered from the current cluster backend.

· One min read
James Chapman

The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

This sprint the teams presented two papers at ICE 2023.

Details​

· One min read
Franco Testagrossa
Pascal Grange

High-level summary​

This week, the Hydra team shared progress updates during the monthly review meeting (monthly report and video recording available soon) and started experimenting on preview network with the new commit from external wallet feature.

What did the team achieve this week​

  • Monthly report & review meeting, demonstrating commit from external wallet
  • Published regular benchmarks for Hydra
  • Moved forward the journey for external commits using multiple script UTxOs #903
  • Changed the API to only put transaction id in snapshots, instead of the full transactions #922 -> this is now evolved into fully addressing #728
  • Fuel marking is now optional as one can now commit from an external wallet #924
  • Add flag option to display node version on tui #934

What are the goals of next week​

  • Complete external commits using multiple script UTxOs #903
  • New release 0.11.0
  • Dirtroad solution of improved persistence performance #913

· 2 min read
Marcin Szamotulski

Network Update​

Key contributions​

We held a series of session to review the implementation of big ledger peers (eclipse evasion). See #4462.

We get a request from a 3rd party to clarify an inconsistency between CDDL spec and protocol implementation. We worked out a nice solution which takes advantage of the any notion available in CDDL. On the Haskell side we provide Any type which gen generate almost any CBOR term (some are excluded only because they are not decoded back to the same form, and we relay on that property). See #4580.

We fixed a bunch of problems of the cardano-cli ping command. It also now has a limited support of node-to-client mini-protocol (the -c option is ignored, as it cannot be supported by node-to-client protocol). Note that the format of messages has changed, timestamps are printed in ISO8601 format. See #4601, #5326, #5313, #30

In order to provide a new flag in the topology file which enables ledger peers when the chain is close to the tip, we continued to work on #4530. This is currently in review, the consensus team will need to provide us with the new api. This feature is useful for two reasons: makes it easier to maintain a topology file, it will also limit the traffic on public roots generated by for example full node wallets and distribute it to ledger peers.

We also continued to work on a blog post which describes the journey of design & implementation of the dynamic P2P network layer. Too be announced soon :).

Other smaller changes​

We limit the concurrency of resolving dns names. Up to 8 root peers or ledger peers DNS names are resolved concurrently, and at most 2 local root peer DNS names. See #4596.

We fixed handshake query timeout in #4603.

We renamed one of the block-fetch decision constructors as requested by the consensus team, see #4608.