Skip to main content

Mithril Team Update

· 2 min read
Jean-Philippe Raynaud
Mithril Tech Lead

High level overview

This week, the Mithril team refactored the unsafe SNARK setup, prepared the prover input for the recursive SNARK aggregation primitives, decoupled the Midnight library in the SNARK recursive circuit, and tested that the degree of the SNARK circuits remains unchanged. They continued improving the modularity of the SNARK recursive circuit using gadgets.

The team also worked on enhancing the aggregator state machine to support the Blocked state and on capturing end-to-end execution logs in artifacts.

Finally, they continued work on shipping the Mithril signer node binary in the Cardano node bundle, releasing the 2628 distribution, upgrading the Cardano node to 11.1, updating the DMQ node to 0.7.0.0, and upgrading the midnight-zk library to 2.3.3.

Low level overview

Features

  • Completed the issue Refactor unsafe SNARK setup #3300
  • Completed the issue Recursive SNARK aggregation primitives: Refactor preparation of prover input #3319
  • Completed the issue Refactor SNARK recursive circuit - Midnight library decoupling #3129
  • Completed the issue Test that SNARK circuits degree are unchanged #3330
  • Worked on the issue Refactor SNARK recursive circuit - Modularity enhancement with gadgets #3131

Protocol maintenance

  • Worked on the issue Ship Mithril signer node binary in Cardano node bundle in GitHub #3011
  • Worked on the issue Release 2628 distribution #3271
  • Worked on the issue Enhance aggregator state machine to support Blocked state #3333
  • Worked on the issue Upgrade to Cardano 11.1 #3346
  • Worked on the issue Update DMQ node to 0.7.0.0 #3358
  • Worked on the issue Capture e2e execution logs in artifacts #3362
  • Worked on the issue Upgrade midnight-zk to 2.3.3 #3375

Plutus Core Team Update

· 2 min read
Ziyang Liu
Software Engineering Lead

High level summary

In the past weeks, the Plutus team has continued to make improvements across performance, tooling, and assurance.

Performance and compiler improvements. We improved several parts of the Plutus optimization pipeline and ledger API implementation. This includes making common Value operations more efficient, adding a new compiler optimization for certain recursive functions, and fixing an optimization pass so that it preserves program behavior correctly.

Better benchmarking and tooling. We added lightweight benchmarking support to the uplc executable, making it easier to sanity-check execution-time measurements for UPLC scripts and compare results against existing benchmark infrastructure.

Specs and formal methods. We continued strengthening the foundations of Plutus by updating the UPLC specification for the built-in value type and related functions, and by improving the metatheory and certification infrastructure used to reason about compiler transformations. These changes make the formal-assurance workflow easier to maintain, debug, and scale to larger programs.

Key Pull Requests Merged

Mithril Team Update

· 3 min read
Jean-Philippe Raynaud
Mithril Tech Lead

High level overview

This week, the Mithril team completed the wiring of the recursive SNARK proof in the aggregate signature, and its integration in the end-to-end tests and its deployment in a test network. They continued work on refactoring the unsafe SNARK setup and refactoring the preparation of the prover input for the recursive SNARK aggregation primitives.

The team also completed the implementation of the monitoring for Cardano blocks and transactions in the protocol dashboard, the integration tests for the unknown signed entity type in the signer and aggregator, and the enhancements of the aggregator state machine to support the Blocked state. They continued work on the release of the 2626 distribution, the upgrade to Cardano 11.1, and the update of the DMQ node to 0.7.0.0.

Finally, the team fixed some flakiness in the Hydra CI, enhanced the versions bump script and implemented a GitHub workflow to automate the marking of stale issues and pull requests.

Low level overview

Features

  • Completed the issue Recursive SNARK aggregation primitives: Wire SNARK proof in aggregate signature #3141
  • Completed the issue Implement recursive SNARK proof generation and verification in end-to-end tests #3142
  • Worked on the issue Refactor unsafe SNARK setup #3300
  • Worked on the issue Recursive SNARK aggregation primitives: Refactor preparation of prover input #3319

Protocol maintenance

  • Completed the issue Implement monitoring for Cardano Blocks and Transactions in Protocol dashboard #3157
  • Completed the issue Activate CardanoBlocksTransactions in Mithril networks #3160
  • Completed the issue Add integration tests for unknown signed entity type in signer and aggregator #3296
  • Completed the issue Flaky CI on hydra run of ci/hydra-build:x86_64-linux.mithril-common #3329
  • Completed the issue Enhance versions bump script #3334
  • Completed the issue Mark stale issues and PRs with GitHub workflow #3339
  • Completed the issue Certification stopped on ivc-snark-preview network #3351
  • Worked on the issue Ship Mithril signer node binary in Cardano node bundle in GitHub #3011
  • Worked on the issue Release 2626 distribution #3271
  • Worked on the issue Enhance aggregator state machine to support Blocked state #3333
  • Worked on the issue How to reduce the IVC tests duration #3343
  • Worked on the issue Upgrade to Cardano 11.1 #3346
  • Worked on the issue Update DMQ node to 0.7.0.0 #3358

Performance & Tracing Update

· 5 min read
Michael Karg
Performance and Tracing Team Lead

High level summary

  • Benchmarking: -O2 build optimization benchmarks.
  • Development: Legacy iohk-monitoring tracing backend removed from cardano-node; RTView removed from cardano-tracer; trace-dispatcher maintenance and ongoing restructuring into Hermod packages.
  • Infrastructure: Genesis cache overhaul with full Protocol Version 11 overlay close to merging.
  • Tracing: ReCon framework and trace-resources moved to Hermod project; cardano-tracer timeseries HTTP API and Grafana datasource merged; ASCII operator spellings for cardano-recon-framework.

Low level overview

Benchmarking

The P&T team has performed benchmarks of a highly optimized build (-O2) vs. a build using standard optimization (-O1), the latter being the current default for our releases. The benchmarks demonstrated that under low to near-idle submission load, no significant gains in resource usage were achieved. Only under full saturation over extended periods of time was a ~17% reduction in Process CPU usage measured, whereas RAM usage was stable. However, in both cases, we observed small increases (3% - 6%) in block adoption times across the network. For the current Praos Node, we still recommend the default. When using another Consensus mode with a significantly higher idle baseline, it's worth noting there's a potential trade-off that can be made.

Development

Last month we reported that the removal of the legacy iohk-monitoring-based tracing backend from cardano-node was complete but awaiting final verification. That PR has now been merged (cardano-node PR#6580). The change eliminates approximately 11,000 lines of dead code across 15 modules and removes 9 transitive build dependencies, leaving trace-dispatcher as the sole tracing backend. With this out of the way, the remaining components of the new tracing system can be moved into the self-contained Hermod Tracing System project repository without coupling their release cycle to cardano-node. Please refer to the PR description for a detailed breakdown of now obsolete Node config keys.

Alongside that, RTView has been removed from cardano-tracer (cardano-node PR#6607). RTView was an experimental, opt-in browser dashboard hidden behind a build flag since version 0.2.4. Its functionality is now fully covered by the Grafana datasource and the timeseries server built into cardano-tracer, making the component redundant. The removal cleans up 35 source files and several build dependencies.

The P&T team has also completed a focused maintenance pass over trace-dispatcher (hermod-tracing PR#12). Additionally, work is underway to restructure trace-dispatcher into two distinct packages — hermod-tracing-api, carrying types and combinators, and hermod-tracing-core, carrying the full backend stack including EKG and Prometheus (hermod-tracing PR#16). This split will allow downstream packages that only need to instrument their own code to avoid pulling in the full backend stack as a dependency.

Infrastructure

The genesis cache overhaul for our performance workbench (the benchmarking automation framework) is completed and close to merging (cardano-node PR#6544). As reported last month, the new approach splits a genesis into two independently-cached halves — the heavy dataset (keys, UTxO entries, delegators, DReps) and the per-era protocol parameters — which are then reassembled on demand. This greatly widens the range of parameters a benchmarking profile can modify without causing a full cache miss. The PR additionally restores non-default Plutus cost models via the Alonzo genesis extraConfig field, and includes a full Protocol Version 11 overlay with the updated cost models and execution budgets that have been submitted as a governance action on mainnet.

Tracing

With the legacy tracing removal merged, the cardano-recon-framework and the trace-resources package have both been moved from cardano-node into the Hermod Tracing System project. The ReCon framework has been accommodated under hermod-tracing (cardano-node PR#6598, hermod-tracing PR#11); trace-resources — which provides cross-platform OS-level resource sampling (CPU, memory, network I/O, filesystem I/O, thread count) — has been renamed hermod-trace-resources (hermod-tracing PR#14), with its module namespace updated from Cardano.Logging.Resources.* to Hermod.Tracing.Resources.*. All logic, platform-specific C bindings, and tests are preserved unchanged.

The cardano-tracer timeseries HTTP API and its accompanying Grafana datasource, which we reported as being in testing last month, have now been merged (cardano-node PR#6562). The API provides PromQL-like queries over metrics timeseries stored by cardano-tracer, aligned closely with the Prometheus HTTP API schema so that existing Grafana integrations and community-built tooling can reuse familiar glue code. The Grafana datasource shipped alongside it replaces the now-removed RTView and includes a reference dashboard for operators to build from.

The cardano-recon-framework formula parsers now accept ASCII alternatives to their Unicode operators (hermod-tracing PR#15). Spellings such as \globally, \finally, && and => are now valid alongside the original Unicode forms, as are numeric indices in parentheses (e.g. \globallyN(2)). This makes authoring LTL properties more accessible in environments where Unicode input is inconvenient, and ships as part of v1.4.0.

Mithril Team Update

· 3 min read
Jean-Philippe Raynaud
Mithril Tech Lead

High level overview

This week, the Mithril team completed the creation of the SNARK proof with the circuit in the recursive SNARK aggregation primitives, the addition of the missing ancillary data for the IVC aggregate signature creation, and the update of the Midnight ZK library audit status for SNARK. They continued work on wiring the SNARK proof in the aggregate signature, refactoring the preparation of the prover input, and refactoring the SNARK setups.

The team also completed the activation of the recursive SNARK prover in a dev network. They continued work on implementing the recursive SNARK proof generation and verification in the end-to-end tests and on reducing the IVC tests duration. They continued work on the integration tests for the unknown signed entity type in the signer and aggregator, the enhancement of the support for discontinued signed entity types in the aggregator, and the upgrade to Cardano 11.1.

Finally, the team completed the upgrade to Pallas 1.1.0 and the enhancement of the versions bump script.

Low level overview

Features

  • Completed the issue Update Midnight ZK library audit status for SNARK #3122
  • Completed the issue Recursive SNARK aggregation primitives: Create SNARK proof with circuit #3139
  • Completed the issue Activate recursive SNARK prover in dev network #3143
  • Completed the issue Add missing ancillary data for IVC aggregate signature creation #3341
  • Worked on the issue Recursive SNARK aggregation primitives: Wire SNARK proof in aggregate signature #3141
  • Worked on the issue Implement recursive SNARK proof generation and verification in end-to-end tests #3142
  • Worked on the issue Refactor unsafe SNARK setup #3300
  • Worked on the issue Recursive SNARK aggregation primitives: Refactor preparation of prover input #3319

Protocol maintenance

  • Completed the issue Upgrade to Pallas 1.1.0 #3297
  • Completed the issue Enhance support for discontinued signed entity types in aggregator #3332
  • Completed the issue Move slow tests to slow tier #3337
  • Worked on the issue Ship Mithril signer node binary in Cardano node bundle in GitHub #3011
  • Worked on the issue Release 2626 distribution #3271
  • Worked on the issue Add integration tests for unknown signed entity type in signer and aggregator #3296
  • Worked on the issue Enhance versions bump script #3334
  • Worked on the issue How to reduce the IVC tests duration #3343
  • Worked on the issue Upgrade to Cardano 11.1 #3346