Skip to main content

Hydra Team Update

· 4 min read
Noon van der Silk
Software Engineering Lead

High-level Summary

The user-friendly hydra-node configuration file has been merged, letting operators run a head from a single YAML file and diff configs against peers at a glance. The rest of the period was mostly hardening: deposit and recovery chain observations are now scoped to the head that produced them (closing a long-standing cross-head contamination issue), the Blockfrost backend retries on HTTP errors instead of crashing and no longer chokes on some inline datums, and the event store's rotation logic got two fixes - a race between VACUUM and streaming client history, and a deadlock in the network layer's outbound message queue that could silently halt broadcasting.

What did the team achieve?

  • Merged the user-friendly hydra-node configuration file: nodes can now be configured via YAML, making it easy to spot differences between peers' setups #2581, #2296
  • Scoped deposit and recover chain observations to the current head: Open/Closed states only record OnDepositTx events for their own headId and Idle silently drops the rest, while recovery of a deposit from a previous head keeps working in every state, including mid-fanout and while a new head is running #2743, #2606
  • Made the Blockfrost chain backend more resilient: HTTP errors are now retried instead of crashing the node #2729, #2477, and a follow-up fixed inline datum decoding that could throw on some UTxOs #2754, #2751
  • Fixed a race between the event log's VACUUM-based rotation and the API server streaming client history over the same SQLite connection, which could take a node down on reconnect #2755
  • Fixed a deadlock in the network layer's PersistentQueue: a false Eq result in popPersistentQueue could leave the broadcast queue stuck at capacity with no log output; it now pops unconditionally by index and emits PersistentQueueFull / PersistentQueueLoadFailed for visibility, and a failed file delete no longer crashes the network component #2742
  • Event log rotation now archives the outgoing database to a numbered old-state/hydra-<logId>.db file instead of deleting it, restoring the old file-based persistence's pre-rotation backup behaviour #2748, #2747
  • Added a fuel verification key option to hydra-tui so the funds tab can show both the head's ada and a separate fuel UTxO #2731
  • Moved the spec back into the hydra repo, dropping the separate flake input #2733

What's next?

  • Merge some in-progress performance work: computing accumulator commitments via the rust-accumulator FFI instead of off-chain PlutusTx, batching etcd broadcasts onto shared connections, and raising the per-snapshot transaction cap - early results show sustained throughput roughly doubling on a 1k-UTxO head #2752
  • Make hydra-chain-observer version-aware by detecting script hashes, so a single process can watch heads from any historical Hydra version instead of running one per version #2740
  • Enforce --deposit-period consistency across head nodes by embedding it in the on-chain OpenDatum #2734
  • Accept PaymentExtendedKey (HD wallet keys) #2583
  • Work on the updated version of the spec (typst+Agda)

Mithril Team Update

· 2 min read
Jean-Philippe Raynaud
Mithril Tech Lead

High level overview

This week, the Mithril team refactored the unsafe SNARK setup, prepared the prover input for the recursive SNARK aggregation primitives, decoupled the Midnight library in the SNARK recursive circuit, and tested that the degree of the SNARK circuits remains unchanged. They continued improving the modularity of the SNARK recursive circuit using gadgets.

The team also worked on enhancing the aggregator state machine to support the Blocked state and on capturing end-to-end execution logs in artifacts.

Finally, they continued work on shipping the Mithril signer node binary in the Cardano node bundle, releasing the 2628 distribution, upgrading the Cardano node to 11.1, updating the DMQ node to 0.7.0.0, and upgrading the midnight-zk library to 2.3.3.

Low level overview

Features

  • Completed the issue Refactor unsafe SNARK setup #3300
  • Completed the issue Recursive SNARK aggregation primitives: Refactor preparation of prover input #3319
  • Completed the issue Refactor SNARK recursive circuit - Midnight library decoupling #3129
  • Completed the issue Test that SNARK circuits degree are unchanged #3330
  • Worked on the issue Refactor SNARK recursive circuit - Modularity enhancement with gadgets #3131

Protocol maintenance

  • Worked on the issue Ship Mithril signer node binary in Cardano node bundle in GitHub #3011
  • Worked on the issue Release 2628 distribution #3271
  • Worked on the issue Enhance aggregator state machine to support Blocked state #3333
  • Worked on the issue Upgrade to Cardano 11.1 #3346
  • Worked on the issue Update DMQ node to 0.7.0.0 #3358
  • Worked on the issue Capture e2e execution logs in artifacts #3362
  • Worked on the issue Upgrade midnight-zk to 2.3.3 #3375

Plutus Core Team Update

· 2 min read
Ziyang Liu
Software Engineering Lead

High level summary

In the past weeks, the Plutus team has continued to make improvements across performance, tooling, and assurance.

Performance and compiler improvements. We improved several parts of the Plutus optimization pipeline and ledger API implementation. This includes making common Value operations more efficient, adding a new compiler optimization for certain recursive functions, and fixing an optimization pass so that it preserves program behavior correctly.

Better benchmarking and tooling. We added lightweight benchmarking support to the uplc executable, making it easier to sanity-check execution-time measurements for UPLC scripts and compare results against existing benchmark infrastructure.

Specs and formal methods. We continued strengthening the foundations of Plutus by updating the UPLC specification for the built-in value type and related functions, and by improving the metatheory and certification infrastructure used to reason about compiler transformations. These changes make the formal-assurance workflow easier to maintain, debug, and scale to larger programs.

Key Pull Requests Merged

Mithril Team Update

· 3 min read
Jean-Philippe Raynaud
Mithril Tech Lead

High level overview

This week, the Mithril team completed the wiring of the recursive SNARK proof in the aggregate signature, and its integration in the end-to-end tests and its deployment in a test network. They continued work on refactoring the unsafe SNARK setup and refactoring the preparation of the prover input for the recursive SNARK aggregation primitives.

The team also completed the implementation of the monitoring for Cardano blocks and transactions in the protocol dashboard, the integration tests for the unknown signed entity type in the signer and aggregator, and the enhancements of the aggregator state machine to support the Blocked state. They continued work on the release of the 2626 distribution, the upgrade to Cardano 11.1, and the update of the DMQ node to 0.7.0.0.

Finally, the team fixed some flakiness in the Hydra CI, enhanced the versions bump script and implemented a GitHub workflow to automate the marking of stale issues and pull requests.

Low level overview

Features

  • Completed the issue Recursive SNARK aggregation primitives: Wire SNARK proof in aggregate signature #3141
  • Completed the issue Implement recursive SNARK proof generation and verification in end-to-end tests #3142
  • Worked on the issue Refactor unsafe SNARK setup #3300
  • Worked on the issue Recursive SNARK aggregation primitives: Refactor preparation of prover input #3319

Protocol maintenance

  • Completed the issue Implement monitoring for Cardano Blocks and Transactions in Protocol dashboard #3157
  • Completed the issue Activate CardanoBlocksTransactions in Mithril networks #3160
  • Completed the issue Add integration tests for unknown signed entity type in signer and aggregator #3296
  • Completed the issue Flaky CI on hydra run of ci/hydra-build:x86_64-linux.mithril-common #3329
  • Completed the issue Enhance versions bump script #3334
  • Completed the issue Mark stale issues and PRs with GitHub workflow #3339
  • Completed the issue Certification stopped on ivc-snark-preview network #3351
  • Worked on the issue Ship Mithril signer node binary in Cardano node bundle in GitHub #3011
  • Worked on the issue Release 2626 distribution #3271
  • Worked on the issue Enhance aggregator state machine to support Blocked state #3333
  • Worked on the issue How to reduce the IVC tests duration #3343
  • Worked on the issue Upgrade to Cardano 11.1 #3346
  • Worked on the issue Update DMQ node to 0.7.0.0 #3358

Performance & Tracing Update

· 5 min read
Michael Karg
Performance and Tracing Team Lead

High level summary

  • Benchmarking: -O2 build optimization benchmarks.
  • Development: Legacy iohk-monitoring tracing backend removed from cardano-node; RTView removed from cardano-tracer; trace-dispatcher maintenance and ongoing restructuring into Hermod packages.
  • Infrastructure: Genesis cache overhaul with full Protocol Version 11 overlay close to merging.
  • Tracing: ReCon framework and trace-resources moved to Hermod project; cardano-tracer timeseries HTTP API and Grafana datasource merged; ASCII operator spellings for cardano-recon-framework.

Low level overview

Benchmarking

The P&T team has performed benchmarks of a highly optimized build (-O2) vs. a build using standard optimization (-O1), the latter being the current default for our releases. The benchmarks demonstrated that under low to near-idle submission load, no significant gains in resource usage were achieved. Only under full saturation over extended periods of time was a ~17% reduction in Process CPU usage measured, whereas RAM usage was stable. However, in both cases, we observed small increases (3% - 6%) in block adoption times across the network. For the current Praos Node, we still recommend the default. When using another Consensus mode with a significantly higher idle baseline, it's worth noting there's a potential trade-off that can be made.

Development

Last month we reported that the removal of the legacy iohk-monitoring-based tracing backend from cardano-node was complete but awaiting final verification. That PR has now been merged (cardano-node PR#6580). The change eliminates approximately 11,000 lines of dead code across 15 modules and removes 9 transitive build dependencies, leaving trace-dispatcher as the sole tracing backend. With this out of the way, the remaining components of the new tracing system can be moved into the self-contained Hermod Tracing System project repository without coupling their release cycle to cardano-node. Please refer to the PR description for a detailed breakdown of now obsolete Node config keys.

Alongside that, RTView has been removed from cardano-tracer (cardano-node PR#6607). RTView was an experimental, opt-in browser dashboard hidden behind a build flag since version 0.2.4. Its functionality is now fully covered by the Grafana datasource and the timeseries server built into cardano-tracer, making the component redundant. The removal cleans up 35 source files and several build dependencies.

The P&T team has also completed a focused maintenance pass over trace-dispatcher (hermod-tracing PR#12). Additionally, work is underway to restructure trace-dispatcher into two distinct packages — hermod-tracing-api, carrying types and combinators, and hermod-tracing-core, carrying the full backend stack including EKG and Prometheus (hermod-tracing PR#16). This split will allow downstream packages that only need to instrument their own code to avoid pulling in the full backend stack as a dependency.

Infrastructure

The genesis cache overhaul for our performance workbench (the benchmarking automation framework) is completed and close to merging (cardano-node PR#6544). As reported last month, the new approach splits a genesis into two independently-cached halves — the heavy dataset (keys, UTxO entries, delegators, DReps) and the per-era protocol parameters — which are then reassembled on demand. This greatly widens the range of parameters a benchmarking profile can modify without causing a full cache miss. The PR additionally restores non-default Plutus cost models via the Alonzo genesis extraConfig field, and includes a full Protocol Version 11 overlay with the updated cost models and execution budgets that have been submitted as a governance action on mainnet.

Tracing

With the legacy tracing removal merged, the cardano-recon-framework and the trace-resources package have both been moved from cardano-node into the Hermod Tracing System project. The ReCon framework has been accommodated under hermod-tracing (cardano-node PR#6598, hermod-tracing PR#11); trace-resources — which provides cross-platform OS-level resource sampling (CPU, memory, network I/O, filesystem I/O, thread count) — has been renamed hermod-trace-resources (hermod-tracing PR#14), with its module namespace updated from Cardano.Logging.Resources.* to Hermod.Tracing.Resources.*. All logic, platform-specific C bindings, and tests are preserved unchanged.

The cardano-tracer timeseries HTTP API and its accompanying Grafana datasource, which we reported as being in testing last month, have now been merged (cardano-node PR#6562). The API provides PromQL-like queries over metrics timeseries stored by cardano-tracer, aligned closely with the Prometheus HTTP API schema so that existing Grafana integrations and community-built tooling can reuse familiar glue code. The Grafana datasource shipped alongside it replaces the now-removed RTView and includes a reference dashboard for operators to build from.

The cardano-recon-framework formula parsers now accept ASCII alternatives to their Unicode operators (hermod-tracing PR#15). Spellings such as \globally, \finally, && and => are now valid alongside the original Unicode forms, as are numeric indices in parentheses (e.g. \globallyN(2)). This makes authoring LTL properties more accessible in environments where Unicode input is inconvenient, and ships as part of v1.4.0.