Skip to main content

Mithril Team Update

· 3 min read
Jean-Philippe Raynaud
Mithril Tech Lead

High level overview​

This week, the Mithril team activated the snark feature by default, completed the prototype of the orchestrator of the setup command, and defined the migration of the existing immutable archives to the deterministic format. They also supported the Kubo RPC API authentication in the aggregator, and published a DNSLink record of the immutable archives IPNS name in the DNS zones of the Mithril networks.

The team kept working on the managed Kubo node download mode and the Kubo RPC API authentication in the client, the discovery of the facts of the Cardano node for the setup command, the prototype of the certification adaptation in the Dijkstra era, and the deployment of a dev-dijkstranet Mithril network.

Finally, they simplified the serialization of the IVC proofs, removed the unused IPFS storage max setting from the infrastructure, and worked on the upgrade to Cardano node 11.2.

Low level overview​

Features​

  • Completed the issue Activate snark feature by default #3573
  • Completed the issue Prototype the orchestrator of the mithril-setup command #3642
  • Completed the issue Skip the PhantomData marker in the serialization of IvcProof #3704
  • Completed the issue Define the migration of the existing immutable archives to the deterministic format #3653
  • Completed the issue Publish a DNSLink record of the immutable archives IPNS name in the Mithril networks DNS zones #3700
  • Completed the issue Support the Kubo RPC API authentication in the aggregator #3657
  • Worked on the issue Implement a managed Kubo node download mode in the client #3655
  • Worked on the issue Support the Kubo RPC API authentication in the client #3656
  • Worked on the issue Discover the facts of the Cardano node for the setup command #3677
  • Worked on the issue Prototype the certification adaptation in the Dijkstra era (before Leios) #3687
  • Worked on the issue Deploy a dev-dijkstranet Mithril network #3689

Protocol maintenance​

  • Completed the issue Remove the unused IPFS storage max setting from the infrastructure #3662
  • Worked on the issue Upgrade to Cardano 11.2 #3585
  • Worked on the issue Update the guild-operators scripts to use the on-chain protocol configuration #3661

Network Team Update

· 13 min read
Marcin Szamotulski
Network Team Lead

Overview​

Summary​

Ouroboros-Network​

We released network-mux-0.12.0.0, ouroboros-network-1.3.0.0 and cardano-diffusion-1.3.0.0, followed by the ouroboros-network 2.0 release, which dmq-node has already adopted. cardano-diffusion 1.1.0.0 and 1.1.1.0 were released earlier.

Tx-submission. The V2 tx-submission without a central decision thread was merged. Since then we fixed duplicate txids in the outbound side, fixed truncated duration counters (waits shorter than a millisecond were counted as zero), switched outbound calculations to Int, and made awaitSharedChange an STM action so dmq-node can resolve races in a single transaction. NodeToNodeV_16 with configurable tx-submission limits was merged. Bounding the work done during a sweep (at most 1000 entries per STM transaction) is in progress.

Network-mux. Mux SDU header encoding/decoding is now about 70% faster and allocates over 90% less. The mux now forces the first 256 KiB of a message in the sender's thread, so an unevaluated encoding waiting in the egress buffer no longer retains whatever it refers to. Mux counters were added, enabling per-mini-protocol dashboards. Fixes to mux timeouts (cancelling timeouts when actions throw and keeping timeout monitors cancellable under masking) are in review, and an RTT-aware mux using echo-cookie RTT sampling is being prototyped. Work has started on a mux egress scheduler, which serves bearers by tiers and tracks egress counters and busy periods, alongside batching changes in the mux (withBatch, GrantAt).

Connection manager & inbound governor. Several race conditions were fixed: connection-manager / inbound-governor races (including a connection leak), a race on new connections in the inbound governor (with a new IOSimPOR test), and the inbound governor no longer blocks on a stale MuxFinished. Two small memory leaks were fixed, in the KeepAlive server and in cardano-node's acceptAcceptor. The diffusion layer now filters configured or negotiated mini-protocols before passing them to the mux.

Peer selection. Peers whose peer-share request failed are now backed off (default 1 hour), and the peer-share time is only updated for peers we are still connected to. Topology reload of local roots given as IP addresses is fixed (previously they waited for DNS-named local roots, never updating when there were none). DNS TTL handling was fixed, and upstreamyness tracing was added, also reducing the likelihood of immediately re-promoting hot peers. We also added a Hashable API, and work on running diffusion on multiple network interfaces is in review.

Ping. The ping API for validating SRV records / registration certificates was merged, to be used by cardano-cli, together with output formatting improvements; further output improvements are in review.

Window-stats. A new window-stats library provides efficient rolling-window statistics backed by a finger tree — a first step towards exposing connection metrics to applications. It already powers the Leios percentile metrics in cardano-node. Follow-ups include API changes, renaming and an insert performance fix.

Smol-world. We are building smol-world, a discrete-event simulator with a realistic transport model (loss, RFC 6298 RTO, CUBIC congestion control per RFC 9438) on a small-world topology with per-node egress contention, to study diffusion at scale. TCP model refinements are in review.

Leios​

The improved ΔQ analysis of Linear Leios EB diffusion was merged. It fixes three bugs in the earlier model, sweeps over EB closure sizes to determine practical limits, and analyses the sensitivity to mempool fragmentation (1-hop vs. multi-hop closure fetch, TxCache miss rate). We also published the incremental decoding benchmark results in the Leios design document. In cardano-node we added CDFs of EB announcement delays (with a follow-up fix) and fixed the Praos CDF; replacing the CDFs with p90/p95/p99 percentiles is in review.

We are also experimenting with a whole-closure store-and-forward EB fetch policy with bounded hedging in ouroboros-consensus: a peer offers a closure only once it holds all of it, receivers request only the missing transactions, and RTT-aware deadlines trigger hedged requests to alternative suppliers when progress stalls. In a 64-node cardano-ignite experiment it reduced median time for a closure to reach all nodes from 5.6 s to 4.1 s, and cut redundant closure payload from 86 MB to 1 MB (amplification 1.53× → 1.01×).

DMQ-Node​

We released dmq-node-0.7.0.0 (ouroboros-consensus-4.0, ouroboros-network-1.2, GHC-9.14 support) and dmq-node-0.7.2.0, and updated to ouroboros-network-2.0 and ouroboros-consensus-5.2. Integration for Node release 11.1 was completed. Other changes include peer-sharing timeouts and an explicit CBOR decode error for local message notification, changes to the signature outbound side, lowering the minimum signature body size to 90 bytes, an internal --min-sig-delay flag, a custom validation config fix, and running Hydra CI on aarch64-linux.

IOSim, io-classes & typed-protocols​

io-classes-1.11 and typed-protocols-1.2.2.0 were released. An io-sim performance fix (reverting eager evaluation of say/traceM/traceSTM arguments) was merged, and ppSayTrace was added. A fix for timeout raising a stale exception after it returned and a TimeDuration type class are in review. IOSimPOR gained finer control over which threads take part in schedule exploration (explorationRacyByLabel, with labelled fork/async variants); this is in review.

Ouroboros-Network​

PR / IssueStatusNotes
Changelog for ouroboros-network 2.0 releasemerged
Release network-mux-0.12, ouroboros-network-1.3, cardano-diffusion-1.3merged
Release cardano-diffusion-1.1.1.0mergedcontribution by f-f
Release cardano-diffusion-1.1.0.0merged
V2 tx-submission without a decision threadmerged
tx-submission configurable limits (NodeToNodeV_16)merged
tx-submission outbound: duplicate txidsmerged
tx-submission: fix truncated duration countersmerged
tx-outbound: Int based calculationsmerged
awaitSharedChange as an STM actionmerged
Tx-submission V2 bounded sweepin progress
Removed partial functions from tx-submission outboundin review
Mux countersmerged
Encode a message's first 256 KiB in sendmerged
More performant mux header encoding / decodingmerged
Mux timeout fixesin review
RTT-aware muxin progress
Mux egress schedulerin progress
mux: withBatch logicin review
Inbound governor changesmerged
Fixed inbound governor race conditionmerged
Prevent some CM/IG racy situationsmerged
Fixed a small memory leak in KeepAliveServermerged
Filter mini-protocolsmerged
Optional JobPool completion queuesmerged
Back off peer-share sources whose request failedmerged
Peer sharing time fixmerged
Fix topology reload of IP address relaysmerged
DNS TTL fixesmerged
Upstreamyness tracingmerged
Hashable APImerged
Drop NodeToClient versionsmerged
Support multiple interfacesin review
Refactored withPeerSelectionActionsin review
Fixed NetworkMagic range check on 32-bit platformsin review
Ping API for registration certificate validationmerged
cardano-ping and formattingmerged
ping: improved outputin review
window-stats: efficient rolling window statisticsin review
Changes to window-stats librarymerged
window-stats changesin review
window-stats: fix insert performancein review
smol worldin progress
smol-world changesin review
BlockFetchConsensusInterface: add matchedBlockmergedcontribution by nfrisby
cardano-diffusion: label ledger peer usagein reviewcontribution by dancewithheart
CI: cache consistencymerged
Added +nightly flag on nightly buildsmerged

Leios​

PR / IssueStatus
Improved ΔQ analysis for Linear Leios EB diffusionmerged
Incremental decoding benchmark resultsmerged
Produce CDF of time to transmit a datum over TCPin progress
CDFs for EB announcementsmerged
Fix for CDF announcement metricsmerged
Fix Praos CDFmerged
Leios statistics (percentiles)in review
Whole closure store and forward, fetch/hedge policyin progress

DMQ-Node​

PR / IssueStatusNotes
Update to ouroboros-network-2.0 and ouroboros-consensus-5.2mergedcontribution by koslambrou
Release dmq-node-0.7.2.0merged
Release dmq-node-0.7.0.0merged
Integration for Node release 11.1mergedcontribution by f-f
sig-outbound changesmerged
Local msg notification codec & peer sharing configurationmerged
Fixed custom validation config bugmerged
Added --min-sig-delay flagmerged
Lower minSigBodySize to 90 bytesmerged
Run Hydra on aarch64-linuxmerged

Cardano-Node​

PR / IssueStatus
Fixed a small memory leak in acceptAcceptormerged

IOSim, io-classes & typed-protocols​

PR / IssueStatus
io-classes-1.11, new release of typed-protocolsmerged
io-sim performance fixmerged
Added ppSayTracemerged
Masked timeout fixin review
Added TimeDuration type classin review
IOSimPOR: racyByLabelin review

Ledger Team Update

· 4 min read
Alexey Kuleshevich
Ledger Team Software Engineer

High level summary​

We continued to focus on Leios this cycle, with the endorser block moving from a data definition to something the ledger can actually validate and apply. The EB and EBLEDGERS rules were implemented, a Leios certificate validation API was exposed from the ledger API, and endorser block application arrived through applyEndorserBlock. Endorser block references were made era-parametric, so they are no longer pinned to a single era. On the pool side, updates that only touch BLS keys no longer require owner witnesses, closing a gap that would have forced unnecessary signatures on key rotation.

Dijkstra gained a distinct mempool transaction representation, separating what the mempool carries from the transaction type used elsewhere. Plutus V4 scripts are now evaluated against the actual V4 API rather than borrowing the previous version's. The state-annotated transaction now also carries reference script size and execution units, so the rules no longer recompute them on every use.

The reference input cost parameter for multi-assets was renamed and retyped, moving from a per-policy to a per-byte cost. Per-policy pricing could not be made fair, since a single policy may carry arbitrarily many asset names, whereas the binary overhead of a multi-asset is straightforward to compute directly.

Testing concentrated on conformance and on the new endorser block machinery. Conformance now tracks which tests are expected to fail, so a disabled test that starts passing is no longer silently ignored, and the tests that already passed were re-enabled. A property test for top-level transaction info was added, comparing the aggregated result against a reference implementation built field by field across the nested transactions of a batch.

Low level summary​

Features​

  • PR-6216 - Leios certificate validation API
  • PR-6215 - Add EB and EBLEDGERS
  • PR-6248 - Add DijkstraApplyBlock with applyEndorserBlock
  • PR-6241 - Make EbReferences era parametric
  • PR-6217 - Do not require owners to witness updates for BLS keys
  • PR-6257 - Introduce BlockEnv for ApplyBlock
  • PR-6260 - Follow-up to #6257: export mkDijkstraBlockEnv, remove mkBbodyEnv
  • PR-6222 - Add MempoolTx
  • PR-6235 - Remove certsTx from SubCertsEnv
  • PR-6255 - Use Plutus v4 API for Plutus v4
  • PR-6244 - Enhanced memoization in StAnnTx
  • PR-6263 - Rename ppRefInputsCostPerMultiAssetByte
  • PR-6251 - Expose PParams from Conway and Dijkstra API

Testing​

  • PR-6237 - Add TopTxInfo property test which compares to reference implementations
  • PR-5984 - Check that disabled tests fail in conformance
  • PR-6240 - Ensure EbSpec test doesn't fail spuriously because of too small txout
  • PR-6256 - Update formal ledger specification

Infrastructure and releasing​

  • PR-6253 - Bump up changelogs and versions after a release
  • PR-6258 - Add Changelog for 11.1 and 11.2

Mithril Team Update

· 2 min read
Jean-Philippe Raynaud
Mithril Tech Lead

High level overview​

This week, the Mithril team stabilized the certificate chain cache in the client library and CLI, and reduced the size of the signature messages to fit within the limit of the DMQ node in testing-preview.

The team kept working on the activation of the snark feature by default, the prototype of the orchestrator of the setup command, and the migration of the existing immutable archives to the deterministic format. They also started working on the discovery of the facts of the Cardano node for the setup command, and on the prototype of the certification adaptation in the Dijkstra era.

Finally, they upgraded Rust to version 1.100, and worked on the upgrade to Cardano node 11.2 and on the update of the guild-operators scripts to use the on-chain protocol configuration.

Low level overview​

Features​

  • Completed the issue Stabilize the certificate chain cache in the client library and CLI #3658
  • Worked on the issue Activate snark feature by default #3573
  • Worked on the issue Prototype the orchestrator of the mithril-setup command #3642
  • Worked on the issue Define the migration of the existing immutable archives to the deterministic format #3653
  • Worked on the issue Discover the facts of the Cardano node for the setup command #3677
  • Worked on the issue Prototype the certification adaptation in the Dijkstra era (before Leios) #3687

Protocol maintenance​

  • Completed the issue Keep the signature messages below the maximum size accepted by the DMQ node in testing-preview #3667
  • Completed the issue Upgrade Rust to version 1.100 #3685
  • Worked on the issue Upgrade to Cardano 11.2 #3585
  • Worked on the issue Update the guild-operators scripts to use the on-chain protocol configuration #3661

Mithril Team Update

· 3 min read
Jean-Philippe Raynaud
Mithril Tech Lead

High level overview​

This week, the Mithril team resolved the SNARK verification key collisions deterministically in the key registration, measured the download time of a mainnet Cardano database with IPFS on dev-mainnet, and implemented a ratio for IPFS downloads in the client. They also filled the fake protocol configuration reader from a JSON hex-encoded marker configuration.

The team kept working on the activation of the snark feature by default, the prototype of the orchestrator of the setup command, the migration of the existing immutable archives to the deterministic format, and the stabilization of the certificate chain cache in the client library and CLI.

Finally, they replaced the retired macos-14 runner in the CI workflows, and worked on the upgrade to Cardano node 11.2 and on the size of the signature messages sent to the DMQ node.

Low level overview​

Features​

  • Completed the issue Resolve SNARK verification key collisions deterministically in the key registration #3591
  • Completed the issue Measure the download time of a mainnet Cardano database with IPFS on dev-mainnet #3608
  • Completed the issue Implement a ratio for IPFS downloads in the client #3641
  • Completed the issue Fill the fake protocol configuration reader from a JSON hex encoded marker configuration #3670
  • Worked on the issue Activate snark feature by default #3573
  • Worked on the issue Prototype the orchestrator of the mithril-setup command #3642
  • Worked on the issue Define the migration of the existing immutable archives to the deterministic format #3653
  • Worked on the issue Stabilize the certificate chain cache in the client library and CLI #3658

Protocol maintenance​

  • Completed the issue Rename DeprecatedCommand to RemovedCommand in the client CLI #3626
  • Completed the issue Replace the retired macos-14 runner in the CI workflows #3645
  • Completed the issue Upgrade the websites JavaScript dependencies to fix security vulnerabilities #3669
  • Worked on the issue Upgrade to Cardano 11.2 #3585
  • Worked on the issue Update the guild-operators scripts to use the on-chain protocol configuration #3661
  • Worked on the issue Keep the signature messages below the maximum size accepted by the DMQ node in testing-preview #3667