Skip to main content

Network Team Update

· 13 min read
Marcin Szamotulski
Network Team Lead

Overview​

Summary​

Ouroboros-Network​

We released network-mux-0.12.0.0, ouroboros-network-1.3.0.0 and cardano-diffusion-1.3.0.0, followed by the ouroboros-network 2.0 release, which dmq-node has already adopted. cardano-diffusion 1.1.0.0 and 1.1.1.0 were released earlier.

Tx-submission. The V2 tx-submission without a central decision thread was merged. Since then we fixed duplicate txids in the outbound side, fixed truncated duration counters (waits shorter than a millisecond were counted as zero), switched outbound calculations to Int, and made awaitSharedChange an STM action so dmq-node can resolve races in a single transaction. NodeToNodeV_16 with configurable tx-submission limits was merged. Bounding the work done during a sweep (at most 1000 entries per STM transaction) is in progress.

Network-mux. Mux SDU header encoding/decoding is now about 70% faster and allocates over 90% less. The mux now forces the first 256 KiB of a message in the sender's thread, so an unevaluated encoding waiting in the egress buffer no longer retains whatever it refers to. Mux counters were added, enabling per-mini-protocol dashboards. Fixes to mux timeouts (cancelling timeouts when actions throw and keeping timeout monitors cancellable under masking) are in review, and an RTT-aware mux using echo-cookie RTT sampling is being prototyped. Work has started on a mux egress scheduler, which serves bearers by tiers and tracks egress counters and busy periods, alongside batching changes in the mux (withBatch, GrantAt).

Connection manager & inbound governor. Several race conditions were fixed: connection-manager / inbound-governor races (including a connection leak), a race on new connections in the inbound governor (with a new IOSimPOR test), and the inbound governor no longer blocks on a stale MuxFinished. Two small memory leaks were fixed, in the KeepAlive server and in cardano-node's acceptAcceptor. The diffusion layer now filters configured or negotiated mini-protocols before passing them to the mux.

Peer selection. Peers whose peer-share request failed are now backed off (default 1 hour), and the peer-share time is only updated for peers we are still connected to. Topology reload of local roots given as IP addresses is fixed (previously they waited for DNS-named local roots, never updating when there were none). DNS TTL handling was fixed, and upstreamyness tracing was added, also reducing the likelihood of immediately re-promoting hot peers. We also added a Hashable API, and work on running diffusion on multiple network interfaces is in review.

Ping. The ping API for validating SRV records / registration certificates was merged, to be used by cardano-cli, together with output formatting improvements; further output improvements are in review.

Window-stats. A new window-stats library provides efficient rolling-window statistics backed by a finger tree — a first step towards exposing connection metrics to applications. It already powers the Leios percentile metrics in cardano-node. Follow-ups include API changes, renaming and an insert performance fix.

Smol-world. We are building smol-world, a discrete-event simulator with a realistic transport model (loss, RFC 6298 RTO, CUBIC congestion control per RFC 9438) on a small-world topology with per-node egress contention, to study diffusion at scale. TCP model refinements are in review.

Leios​

The improved ΔQ analysis of Linear Leios EB diffusion was merged. It fixes three bugs in the earlier model, sweeps over EB closure sizes to determine practical limits, and analyses the sensitivity to mempool fragmentation (1-hop vs. multi-hop closure fetch, TxCache miss rate). We also published the incremental decoding benchmark results in the Leios design document. In cardano-node we added CDFs of EB announcement delays (with a follow-up fix) and fixed the Praos CDF; replacing the CDFs with p90/p95/p99 percentiles is in review.

We are also experimenting with a whole-closure store-and-forward EB fetch policy with bounded hedging in ouroboros-consensus: a peer offers a closure only once it holds all of it, receivers request only the missing transactions, and RTT-aware deadlines trigger hedged requests to alternative suppliers when progress stalls. In a 64-node cardano-ignite experiment it reduced median time for a closure to reach all nodes from 5.6 s to 4.1 s, and cut redundant closure payload from 86 MB to 1 MB (amplification 1.53× → 1.01×).

DMQ-Node​

We released dmq-node-0.7.0.0 (ouroboros-consensus-4.0, ouroboros-network-1.2, GHC-9.14 support) and dmq-node-0.7.2.0, and updated to ouroboros-network-2.0 and ouroboros-consensus-5.2. Integration for Node release 11.1 was completed. Other changes include peer-sharing timeouts and an explicit CBOR decode error for local message notification, changes to the signature outbound side, lowering the minimum signature body size to 90 bytes, an internal --min-sig-delay flag, a custom validation config fix, and running Hydra CI on aarch64-linux.

IOSim, io-classes & typed-protocols​

io-classes-1.11 and typed-protocols-1.2.2.0 were released. An io-sim performance fix (reverting eager evaluation of say/traceM/traceSTM arguments) was merged, and ppSayTrace was added. A fix for timeout raising a stale exception after it returned and a TimeDuration type class are in review. IOSimPOR gained finer control over which threads take part in schedule exploration (explorationRacyByLabel, with labelled fork/async variants); this is in review.

Ouroboros-Network​

PR / IssueStatusNotes
Changelog for ouroboros-network 2.0 releasemerged
Release network-mux-0.12, ouroboros-network-1.3, cardano-diffusion-1.3merged
Release cardano-diffusion-1.1.1.0mergedcontribution by f-f
Release cardano-diffusion-1.1.0.0merged
V2 tx-submission without a decision threadmerged
tx-submission configurable limits (NodeToNodeV_16)merged
tx-submission outbound: duplicate txidsmerged
tx-submission: fix truncated duration countersmerged
tx-outbound: Int based calculationsmerged
awaitSharedChange as an STM actionmerged
Tx-submission V2 bounded sweepin progress
Removed partial functions from tx-submission outboundin review
Mux countersmerged
Encode a message's first 256 KiB in sendmerged
More performant mux header encoding / decodingmerged
Mux timeout fixesin review
RTT-aware muxin progress
Mux egress schedulerin progress
mux: withBatch logicin review
Inbound governor changesmerged
Fixed inbound governor race conditionmerged
Prevent some CM/IG racy situationsmerged
Fixed a small memory leak in KeepAliveServermerged
Filter mini-protocolsmerged
Optional JobPool completion queuesmerged
Back off peer-share sources whose request failedmerged
Peer sharing time fixmerged
Fix topology reload of IP address relaysmerged
DNS TTL fixesmerged
Upstreamyness tracingmerged
Hashable APImerged
Drop NodeToClient versionsmerged
Support multiple interfacesin review
Refactored withPeerSelectionActionsin review
Fixed NetworkMagic range check on 32-bit platformsin review
Ping API for registration certificate validationmerged
cardano-ping and formattingmerged
ping: improved outputin review
window-stats: efficient rolling window statisticsin review
Changes to window-stats librarymerged
window-stats changesin review
window-stats: fix insert performancein review
smol worldin progress
smol-world changesin review
BlockFetchConsensusInterface: add matchedBlockmergedcontribution by nfrisby
cardano-diffusion: label ledger peer usagein reviewcontribution by dancewithheart
CI: cache consistencymerged
Added +nightly flag on nightly buildsmerged

Leios​

PR / IssueStatus
Improved ΔQ analysis for Linear Leios EB diffusionmerged
Incremental decoding benchmark resultsmerged
Produce CDF of time to transmit a datum over TCPin progress
CDFs for EB announcementsmerged
Fix for CDF announcement metricsmerged
Fix Praos CDFmerged
Leios statistics (percentiles)in review
Whole closure store and forward, fetch/hedge policyin progress

DMQ-Node​

PR / IssueStatusNotes
Update to ouroboros-network-2.0 and ouroboros-consensus-5.2mergedcontribution by koslambrou
Release dmq-node-0.7.2.0merged
Release dmq-node-0.7.0.0merged
Integration for Node release 11.1mergedcontribution by f-f
sig-outbound changesmerged
Local msg notification codec & peer sharing configurationmerged
Fixed custom validation config bugmerged
Added --min-sig-delay flagmerged
Lower minSigBodySize to 90 bytesmerged
Run Hydra on aarch64-linuxmerged

Cardano-Node​

PR / IssueStatus
Fixed a small memory leak in acceptAcceptormerged

IOSim, io-classes & typed-protocols​

PR / IssueStatus
io-classes-1.11, new release of typed-protocolsmerged
io-sim performance fixmerged
Added ppSayTracemerged
Masked timeout fixin review
Added TimeDuration type classin review
IOSimPOR: racyByLabelin review