Skip to main content

· 3 min read
Michael Karg

High level summary

  • Benchmarking: 10.4.1 release benchmarks; UTxO-HD, GC settings and socket I/O feature benchmarks.
  • Development: Abstracting over quick queries and trace queries; enabling query processing on remote hosts.
  • Infrastructure: Workbench simplification merged; GHC8.10 tech debt removed.
  • New Tracing: Provided hotfix for several metrics.

Low level overview

Benchmarking

We've completed release benchmarks for Node 10.4.1. It is the first mainline release of an UTxO-HD node featuring LedgerDB. Leading up to the release, we previously performed and analysed UTxO-HD benchmarks. We were able to document a regression in RAM usage, and assisted in pinpointing its origin, leading to it being fixed swiftly for the 10.4 release.

Additionally, we ran feature benchmarks for a potential socket I/O optimization in the network layer, and GC setting changes catering to the now-default GHC9.6 compiler. Both benchmarks have shown moderate improvements in various performance metrics. This might enable the network team to pick up the optimization for 10.5. Also, we might be able to update the recommended GC settings for block producers, and add them to our own nix service configs for deployment.

The 10.4.1 performance report has been published on Cardano Updates.

Development

We've further evolved the (still experimental) quick query feature of our analysis tool locli. Parametrizable quick queries allow for arbitrary queries into raw benchmarking data, uncovering correlations not part of standard analysis. They are implemented using composable definitions inside a filter-reduce framework. With locli's DB storage backend, we can leverage the DB engine to do much of the work. Now, we're integrating a precursor to quick queries - so called trace queries - into the framework. Those can process raw trace data from archived log files. Currently, we're adding an abstraction layer such that it is opaque to the framework whether the data was retrieved (and possibly pre-processed) from a DB or from raw traces.

Furthermore, we added a custom (CBOR-based) serialization for intermediate results so a query can be evaluated on a remote machine - like the system archiving all benchmarking runs - but triggered, and its results visualized, on your localhost.

Infrastructure

The workbench nix code optimization has finally been merged. Redundant derivations and recursions have been replaced; many nix store entries have been consolidated. Among other things, the new code also aims to maximize nix cache hits. Furthermore, as GHC8.10 has now been officially retired from all build pipelines, we were able to clean up all tech debt in our automations that we had to keep around due to supporting the old compiler version.

Exactly as we had hoped, this has brought down CI time for the Node by orders of magnitude; first, from over an hour to around 15 min, then to under 10 min. Also, all workbench shell invocations are significantly faster, and clutter in the nix store is greatly reduced.

New Tracing

We've been hurrying to provide hotfixes for connectionManager_* and slotsMissed metrics that were faulty on Node 10.3. They have been successfully integrated into the Node 10.4 release.

· One min read
Damian Nadales

High level summary

  • Updated the centralized chain-sync-jumping model to allow using it as a substitute for the real model, which will make it easier to troubleshot potential test failures found in our property tests (#1466).
  • Engaged with Cardano Foundation on the Antithesis workstream, which will help us widen the test coverage and robustness of the Cardano node.
  • Enriched db-immutaliser to inspect a VolatileDB (1482).
  • Released Node 10.4.1, which incorporates the UTXO-HD feature. The default backend will be the in memory one. The LMDB backed is not recommended for block producers, but we encourage users to experiment with it and report their experience.
    • Our benchmarks show no significant performance degradation, rating it production ready.
      • A small 2% / 5% (169MiB / 389MiB) increase in heap size could be expected.
      • However, this is mitigated by the fact that going from 10.2 to 10.3, we saw a significant reduction in heap size - namely 6% / 14% (579MiB / 1267MiB).
      • 10.4 reduces CPU usage slightly again
      • There is no impact on network performance.

· 2 min read
Jean-Philippe Raynaud

High level overview

This week, the Mithril team released the pre-release version of the 2517.0-pre distribution. This version supports Cardano node v.10.3.1 and allows for signing ancillary files in the Cardano database snapshots using an IOG key. It also includes various bug fixes and improvements. The team continued implementing incremental certification for the Cardano database, also known as Cardano node database v2, which has been activated on the release-preprod test network.

The team has completed the proof of concept for the Mithril/ALBA aggregation proof system. They also simplified the configuration of the aggregator's commands and improved the dependency injection process. In addition, they implemented a publisher abstraction to facilitate the publication of signatures in the signer.

Finally, the team focused on speeding up the compilation time of Rust nodes and libraries, while continuing to refactor the STM cryptographic library for improved clarity.

Low level overview

  • Pre-released the new distribution 2517.0-pre
  • Completed the issue Sign ancillary files with IOG key #2362
  • Completed the issue Mithril/ALBA aggregation proof system PoC #2403
  • Completed the issue Implement a publisher abstraction for publication of signatures in signer #2427
  • Completed the issue Simplify dependency injection container structure in aggregator #2366
  • Completed the issue Simplify the configuration of commands in aggregator #2384
  • Completed the issue Accelerate Rust compilation time with workspace dependencies #2430
  • Completed the issue Hydra CI is flaky #2436
  • Worked on the issue Release 2517 distribution #2410
  • Worked on the issue Rollout Cardano Database v2 feature #2409
  • Worked on the issue Upgrade to Cardano 10.3 #2446
  • Worked on the issue Re-organize STM library structure #2369
  • Worked on the issue Signer fails to access Cardano node after it has rebooted #2426
  • Worked on the issue Fix aggregator commands automatic documentation #2450

· 2 min read
Ziyang Liu

High level summary

On the Plinth side, we added strictness analysis in both UPLC and PIR. This is used by two compiler optimizations: inlining, and the strictification of bindings For example, if a non-strict binding is guaranteed to be evaluated, it can be safely converted to a strict binding. These optimizations reduce both the computational cost and the size of the resulting Plutus scripts, sometimes significantly.

In preparation for the next hard fork, development on UPLC has progressed steadily. We completed the specification for the dropList primitive and conformance testing for builtin arrays. Additionally, we advanced work on casing on booleans and the costing of the modular exponentiation primitive - both challenging tasks that we expect to complete in the coming weeks.

On the certified compilation front, we expanded our test coverage, addressed several issues in the decision procedures, and are actively working to resolve the remaining recently identified problems.

Low level summary

Key Pull Requests Merged

Issues and Pull Requests In Progress

  • Allow casing on booleans
  • Costing of builtin functions operating on builtin arrays (in plutus-private)
  • Costing of the modular exponentation primitive (in plutus-private)
  • Conformance tests for the modular exponentation primitive (in plutus-private)

· One min read
Noon van der Silk

High-level summary

We're very happy to have release 0.21.0; featuring many big changes. Check out the full release notes here: Hydra 0.21.0. We continue working on supporting the Glacier Drop; the Blockfrost chain layer, fixing some bugs in the incremental commit workflow (please consider this feature in beta until these are fixed), among other day-to-day improvements.

What did the team achieve?

  • Released version 0.21.0! With a tonne of new features.
  • Completed example of withdraw-zero based sha512 trick #1933.
  • Progress on Blockfrost chain layer #1305.
  • Investigated a feature to "Re-open" a closed head; decided against implementing it for security reasons.
  • Continued work on fixes to the deposit workflow #1951

What's next?

  • Finished new deposit check #1951
  • Fix "UX" bug when changing peers on a previously-configured hydra-node #1937
  • Work on a simple endpoint to check head status #1957
  • Continued work on a full Blockfrost chain layer #1305
  • Reinstate Kupo support #1785
  • Investigate bug allowing fan-out #1476
  • Investigate event-log rotation #1581
  • Continue to focus on supporting the Glacier drop