Skip to main content

· 4 min read
Michael Karg

High level summary

  • Benchmarking: Release benchmarks for 10.5; LMDB (UTxOs on-disk) benchmarks.
  • Development: New memory-constrained benchmark families, targeting UTxOs on-disk.
  • Infrastructure: Migration of bench cluster completed.
  • Tracing: Deprecation of legacy system; TCP forwarding merged; improved self-documentation.
  • Meetup: Performance & Tracing meetup held in Cardiff, Wales.

Low level overview

Benchmarking

We've performed and analysed release benchmarks for Node 10.5. The pre-release turned on peer sharing by default; our benchmarks indicated a negative performance impact when enabling this on block producers. The current release 10.5.1 does not enable peer sharing for block producers; the published final results can be found in the Performance report for 10.5.

Additionally, we've achieved meaningful benchmarks for UTxOs-on-disk, which use the LMDB backend. Our new profiles support seamless scaling of RAM pressure on the backend, forcing disk I/O to a varying degree. We're currently analysing the observations made, and gathering more data using different scaling factors if needed; the goal is a reliable assessmemt of LMDB's viability for block producing nodes.

Development

Developing RAM-constrained benchmarks that would put tunable pressure on an UTxOs-on-disk backend posed a unique challenge.

First and foremost, limiting memory for past in-memory benchmarks has never been a requirement at all. A consistent approach to do so given the existing deployment had to be built, along with pertinent diagnostic tooling. Second, the LMDB backend is not managed by Haskell's GHC runtime, but comes with its own memory management - which required us to develop a double-pronged approach to selectively apply RAM limits. Lastly, other parts of the Node's code didn't support executing in tightly limited, constant space and would lead to the OS terminating the Node for running out of memory.

The interaction of various cgroup limits on Linux kernels, memory settings in our Nomad deployment and GHC RTS options let us create a stable runtime behavior over the course of the benchmark - a hard requirement, as system metrics taken at the beginning of a run must be comparable to those towards the end. A blocker for initializing the Node for benchmarks was resolved in cardano-node PR#6295: Using mmap allowed us to use the OS's virtual memory subsystem for on-demand loading instead of it being managed by the Haskell runtime - which significantly brought down the heap size required for the task.

Infrastructure

The migration to the new benchmarking cluster is finalized. For representative performance measurements of UTxOs-on-disk, we require direct SSD storage on each cloud machine instance. Along with deploying the new hardware came a full upgrade of OS and nixpgks software. Validating the migration was extra effort: A seemingly innocent cloud provider service (which was newly added to their NixOS image) did nothing more than a heartbeat request every 5 min to some central infrastructure server. Yet, it caused the standard deviation of some of our network related metrics to double - thus reducing confidence in those metrics.

After validation, we performed a complete re-run of existing performance baselines on the new hardware.

Tracing

Work on the new tracing system has yielded various improvements. Trace forwarding over TCP is now fully functional and merged to master. This will make setting up forwarding to remote hosts much easier than by using UNIX domain sockets / Windows named pipes. However, it's recommended for secure and isolated environments only (cardano-node PR#6241).

The auto-documentation feature of the new system has improved user experience; the generated documents are now structed in a more accessible way, and contain necessary metadata as to which Node version / commit is being documented by them (cardano-node PR#6283).

Another refactoring targeted the new system's core library, trace-dispatcher. It now comes with a minimal dependency footprint and fully equipped with all necessary type definitions to be built using Haskell's community packages (Hackage) exclusively. This greatly enhances the ease to use it for other applications than the Node as well - Cardano or non-Cardano (cardano-node PR#6268). Increasing the dependency footprint is only required for additional features, like trace forwarding.

With the upcoming Node 10.6 release, we plan to officially deprecate the legacy tracing system. This means it will enter a grace period of ~3 months, where both systems coexist in the same Node build; then, it will be decommissioned and actively removed from the code base.

Meetup

We organized 2025's in-person team meetup in Cardiff, Wales. We had highly insightful and productive days - I would like to thank all team members who contributed, and extend my special thanks to guests from outside of the team: Your presence and contributions were greatly appreciated, Can Huzmeli and Neil Davies.

· 2 min read
Noon van der Silk

High-level summary

We have been extremely busy these last few weeks supporting the glacier drop. We continue to focus on this, as well as starting to make some broader changes.

What did the team achieve?

  • Released several patch versions; latest being 0.22.4

    • On this branch we have made many improvements:
    • Fix Hydra API not correctly interpreting events after event log rotation #2170
    • Ignore snapshot signatures of already confirmed snapshots; reduces load when running mirror nodes #2146
    • Fixed internal persistent queue blocking when it reached capacity #2147
    • Address etcd blocking on writes when under load #2171
    • Fixed log rotation id consistency #2173
    • Fix etcd lease keep-alive logic when the lease expired #2148
    • Fix InvalidTx causing stuck head #2134
    • Identified mirror nodes as a solution to node crashing and failing to restart #2117
  • On master

    • Added HTTP endpoint for submitting a transaction #2107
    • Allow partial Ada commit during incremental commits #2140
    • Hydra API server responds with content-type application/json #2094
    • Greetings message contains more information #2087
    • Fixed bug around fanning out with decommit/increment in progress
    • Hydra node rejects UTxO if it's below the limit #2106
    • Fixed log rotation id consistency #2173
    • Fix InvalidTx causing stuck head #2134

What's next?

  • "Forward-port" the changes from 0.22.4 to master
  • Resolve the TUI being out of sync due to event-log rotation #2156
  • Start to investigate implementing partial fanout
  • Enable recovering a deposit when the head is closed #1812
  • Working on CI for Mac #2139
  • Working on a log viewer #2047
  • Present hydra doom at RareEvo
  • Continue to support the glacier drop

Thanks

Big thanks to Juan and Jonathan for big contributions to these recent releases!

· One min read
Damian Nadales

High level summary

During the past couple of weeks, the team has continued working across multiple workstreams: LSM-tree integration, Peras implementation, KES agent integration, and Leios prototyping. Current efforts include:

  • Implementing a streaming conversion for ledger snapshots.
  • Drafting a set of mini-protocols to cooperatively implement the Linear Leios specification. Particular focus is being placed on ensuring adequate concurrency and resource usage bounds, which are less explicitly defined in the high-level spec.
  • Releasing a stubbed version of kes-agent, and integrating it into the Consensus version targeted for Cardano Node 10.6. While the KES code will be included in this release, functionality will remain inactive pending a code audit.
  • Merging Peras-related optimizations into both the main and peras-staging branches.

· 2 min read
Ziyang Liu

High level summary

Plutus Core

  • Following the support for case analysis on booleans, integers and lists, we are now working on supporting case analysis for unit, pairs, and Data.

  • The required changes to enable all built-in functions, as well as sums-of-products, in all Plutus ledger languages (Plutus V1, V2 and V3) have been implemented in preparation for the upcoming intra-era hard fork.

  • We started developing a surface language benchmark for UPLC, aimed to compare the performances of different surface languages targeting UPLC. We have introduced the first scenario for comparison - computing Fibonacci numbers. More scenarios will be added in the near future.

Plinth

  • The Plinth compiler now compiles Haskell Bool into UPLC's built-in boolean type, instead of sums-of-products or Scott encoded boolean. This makes many scripts smaller and faster.

  • We are planning additional performance and usability improvements, such as inlining recursive bindings, and showing line numbers in compilation errors.

Low level summary

Key Pull Requests Merged

Issues and Pull Requests In Progress

· 2 min read
Marcin Szamotulski

Overview of sprint 93

Tx-Submission

We deployed our tx-submission work (see Issue#3311 based on the cardano-node-10.5 release. The node has been working without any problems.

Mithril

We prepared two PR which will allow us to capture raw bytes alongside decoded data structures. This is essential for checking signatures as well as avoiding re-serialisation of signed data:

These PRs are in review.

We now have a dmq node with the following capabilities:

  • can disiminate signatures, but not validate them in any way, although one cannot submit signatures (/work in progress/)
  • discover peers (with peer-sharing mini-protocol)
  • control connections through keep-alive mini-protocol
  • configuration file which allows setting diffusion options, setting up traces, etc.
  • basic logging system

In the current state of development, we were able to connect two dmq nodes and observe keep-alive messages. In the next stage, we will be able to submit signatures and observe how they are diffused through a network of dmq nodes.

Connection Duration

We added a trace point which allows one to see how long a connection lived, see PR#5146.

Release of ouroboros-network-0.22.1

We released ouroboros-network-0.22.1. It is already integrated with ouroboros-consensus and it will be included in cardano-node-10.6. See release-board.