Skip to main content

47 posts tagged with "network"

View All Tags

· 2 min read
Marcin Szamotulski

Overview of sprint 80 & sprint 81​

Current workstream​

We decided to hold some PRs in favour of some others to simplify the merging process. Here's a dependency graph for PRs on which we're working on. The Extensible Ouroboros Network Diffusion Stack PR was the largest in our queue.

Fixes which made into the on-going cardano-node-10.2.1​

Previously, we introduced a new IOError handling policy due to a discussion with an SPO to make some scenarios easier to debug. After testing it (the changes never made it into a release), it turned out this could lead to attacks on the system. Thus, we advocate for better monitoring of nodes (e.g. if resources like file descriptors memory are available, the node is making progress) rather than rely on cardano-node to be up and running.

New mux strategy for starting mini-protocols​

Karl Knutsson (CF) implemented a new strategy for starting mini-protocols, StartOnDemandAny. A mini-protocol which is using this strategy will be started as soon as any StartOnDemand (or StartOnDemandAny) mini-protocol receives input from the network. We will use this starting strategy for the keep-alive mini-protocol.

Local-TX-Monitor protocol changes​

A new query was added, which allows the retrieval of all measures/dimensions of the mempool capacity, e.g. byte-size capacity, ledger's execution units for both memory and execution steps, and reference scripts size. See ouroboros-network#4918.

Network Specification Updates​

We made language improvements in the network specification, see ouroboros-network#5044; and some smaller changes/fixes ouroboros-network#5053.

Other minor changes​

· 4 min read
Marcin Szamotulski
Karl Knutsson

Overview of sprint 78 & sprint 79​

Documentation​

We reviewed the technical report. We closed a number of issues, the most important are:

And a few smaller issues:

This was done in the Network Spec Update PR, we also fixed many grammar & spelling errors network-spec: language.

SRV Record Support​

We worked on SRV records support by ouroboros-network, issue #2780, PR 5018. We will merge it after reuasble diffusion.

Quering Network State through Node-to-Client Protocol​

The aim is to make P2P network stake available through the Local State Query Mini-Protocol.

We opened a draft PR, also see the issue, where we mentioned all the branches where the work is progressing. See below for more technical details.

Extensible Ouroboros Network Diffusion Stack​

The work stream reached the review phase. See issue#5016.

Tx-Submission​

The Consensus team agreed to implement needed mempool performance optimisations and is making progress on them. See ouroboros-consensus#1359.

Ouroboros-Network-0.19 Release​

We cut ouroboros-network-0.19 and 0.19.1 releases.

Configuration Changes for Block Propagation Times​

Block propagation times are influenced by the number of TCP round trips required to transmit a block.

In mid-December, we published a post discussing configuration changes to the Linux IP stack. These adjustments involved increasing the initial TCP congestion window to 42 segments and ensuring that the congestion window remained open for idle connections.

IOG applied these changes to four stake pools located in Brazil, South Africa, Dubai, and Japan around December 15th.

The Cardano Foundation manages a standard peer-to-peer (P2P) node in Paris, which operates without manual connections to other Cardano Foundation nodes or IOG nodes. After implementing the configuration changes, we noted a statistically significant improvement in the propagation times for blocks larger than 10 segments (about 14,480 bytes) produced by IOG's pools.

Block Size (bytes)Improvement (ms)
14,480 - 28,960-132 to -78
28,960 - 57,920-197 to -130
>57,920-255 to -176
Block Propagation Times

These results demonstrate that a Stake Pool Operator (SPO) can enhance the propagation times of their own pool's blocks by applying config changes targeting TCP's congestion window.

Low-level summary​

Querying Network State through Node-to-Client Protocol​

In the first interaction, we will make it possible to query the node-to-node state through LocalStateQuery mini-protocol (part of the node-to-client protocol).

data ConnectionManagerState peeraddr = ConnectionManagerState {
    connectionMap                 :: Map (ConnectionId peeraddr) AbstractState,
    -- ^ map of connections, without outbound connections in
    -- `ReservedOutboundSt` state.

    registeredOutboundConnections :: Set peeraddr
    -- ^ set of outbound connections in the `ReserverdOutboundSt` state.
  }
  deriving (Eq, Show)


data InboundState peeraddr = InboundState {
    remoteHotSet  :: !(Set (ConnectionId peeraddr)),
    remoteWarmSet :: !(Set (ConnectionId peeraddr)),
    remoteColdSet :: !(Set (ConnectionId peeraddr)),
    remoteIdleSet :: !(Set (ConnectionId peeraddr))
  }
  deriving (Eq, Show)

data OutboundState peeraddr = OutboundState {
    coldPeers :: Set peeraddr,
    warmPeers :: Set peeraddr,
    hotPeers  :: Set peeraddr
  }
  deriving (Eq, Show)


data NetworkState peeraddr = NetworkState {
    connectionManagerState :: ConnectionManagerState peeraddr,
    inboundGovernorState   :: InboundState peeraddr,
    outboundGovernorState  :: OutboundState peeraddr
  }
  deriving (Eq, Show)

· 2 min read
Marcin Szamotulski

Overview of sprint 77​

Initiator only mode for local roots​

We implemented the initiator-only mode for local roots peers described in ouroboros-network#5020. This feature will be available in cardano-node-10.3 (cardano-node#6055).

One will be able to specify diffusionMode (either InitiatorOnly or IntiatorAndRespoinder, the latter is the default) for all local roots in a given local roots group, e.g.

{ "localRoots":
    [ { "accessPoints":
          [ { "address": "10.0.0.1"
            , "port": 3001
            }
          ]
      , "advertise": false
      , "diffusionMode": "InitiatorOnly"
      , "warmValency": 1
      , "hotValency": 1
      }
    , { "accessPoints":
          [ { "address": "10.0.0.2"
            , "port": 3001
            }
          ]
      , "advertise": true
      , "diffusionMode": "InititiatorAndResponder"
      , "warmValency": 1
      , "hotValency": 1
      }
    ]
, "publicRoots": []
, "useLedgerAfterSlot": -1
}

As part of ouroboros-network#5020 we had to change how connections are identified in the simulated testnet environment. We exposed ConnStateIdSupply through P2P interfaces, which allows us to use a global ConnStateIdSupply for all nodes in the simulation. This way, ConnStateId is unique for each connection in the simulation. See ouroboros-network#5026.

Support systems with multiple IP addresses​

We merged ouroboros-network#5017 which allows to run cardano-node on systems with multiple network interfaces or a single interface with multiple IP addresses.

Reusable diffusion​

We were working on early integration of reusable diffusion work stream with ouroboros-consensus & cardano-node. Reusable diffusion will allow us to support both cardano-node and mithril-node in the future. We are pleased to say that we are running a cardano-node that uses refactored diffusion. See:

Tx-Submission Logic​

We had a discussion with the consensus team (Karl Knutsson CF, Nick Frisby Tweag) on network requirements for tx-mempool. See:

SRV record support​

We continued working on the SRV support, see:

Technical Debt​

We renamed some of the test modules to be more consistent across various network components, see ouroboros-network#5028.

· 3 min read
Marcin Szamotulski

Overview of sprint 76​

Network Load​

The Cardano Foundation monitors the network by injecting "canary" transactions, allowing us to measure the time taken for a transaction to be included in a block. Approximately four to five of these transactions are submitted every hour.

Since early November, network load has increased, but most transactions are still included within the first or second block after submission. Even if the network is not congested, transactions may not reach the Block Producer responsible for the next block in time due to network propagation delays. Therefore, only a transaction that takes three or more blocks to be included indicates a loaded network.

Since November 1, 2024, we have submitted around 3,700 "canary" transactions, and only 59 of them took longer than two blocks to be included, meaning 98.4% of our test transactions were included within the two-block limit. During this period, the longest observed time to inclusion was five blocks for two transactions and the maximum wait time is 201 seconds.

Network Load

Where the y-axis is a fraction of the block size in the respected dimension (e.g. size, steps, memory).

Hot Standby Block-Producer Nodes​

In the last Network Working Group Meeting (NWG) we discussed solutions for running hot standby block producers using P2P.

Short-Term Solution: Local Root Initiator Only Outbound Connections​

We agreed to implement initiator-only connections for local roots, which wouldn't bind to the node address (and thus would be made from an ephemeral port). This allows SPOs to have hot standby nodes which are hidden from relays using a firewall rule, while they still can connect to the relays (so they receive blocks while the blocks they produce are not sent to the network). See the ouroboros-network#5020 issue and associated PRs:

The latter PR makes P2P node run on systems with multiple network interfaces or a single interface with multiple IP addresses.

An SPO can control the diffusion mode (initiator only / initiator and responder) in the topology file. A draft PR is available in the coot/connection-manager-initiator-only-0.13.2.5 branch.

In the NWG meeting, it was agreed that once this functionality is implemented, we can remove NonP2P code from ouroboros-network (ouroboros-network#5007).

Long-Term Solution​

An agreed long-term solution is to allow to control block propagation of self-produced blocks through cardano-cli.

Reusable Diffusion​

We continued working on making ouroboros-network reusable for both cardano-node and mithril (see CIP#137). The PR can be found here: ouroboros-network#5016, while a more detailed description of its direction can be found here. During this sprint, we carried out a team-wise PR review.

Block Fetch Client For Genesis​

A new block-fetch client designed for Genesis was merged, see ouroboros-network#4919. The design of the new block-fetch Gensis logic is described in here.

KES Agent: Raw Bearer API​

We recently merged ouroboros-network#4395 - a PR which is important for the KES-agent workstream.

· 2 min read
Marcin Szamotulski

Overview of sprint 74​

Dropping NonP2P network stack​

We are advertising that we will drop the non-p2p network stack. This will happen after the next hard fork. Please comment on the PR if this will be problematic for you. Here's the PR: ouroboros-network#5007.

Node-to-Client versions​

We dropped node-to-client versions earlier than version 16, ouroboros-network#5002.

Mithril Design​

We completed review how to adopt ouroboros-network for Mithril design. We started to experiment with ouroboros-network in that direction.

Next Cardano-Node release​

The next cardano-node release will include the following changes from the networking team:

Typed-Protocols​

The integration of ouroboros-network-0.18, which is using the new typed-protocols-0.3 with ouroboros-consensus, was merged, ouroboros-consensus#1223. Integration with cardano-cli and cardano-api was rebased and handed to the release engineer. Integration with ekg-forward was merged: ekg-forward#35.

Genesis​

The following changes to support Genesis (both creating big ledger peer snapshots & providing them in a topology file)

Connection Manager​

We merged the documentation review of the connection manager, ouroboros-network#5001. We also renamed connection manager methods (as suggested by Neil Davies ([PNSOL])): ouroboros-network#5000.

Outbound Governor​

We started to experiment with using FRP to implement the outbound governor in a way that is both easier to maintain and reason about.