46 posts tagged with "network"

Network Update (Sprints 39 & 40)​

The network updates got distracted. Mostly because of me leaving for the meeting in Edinburgh, Scotland and the being away. It was great to see and talk to so many of the community members.

Sprint 39 (Jun 26 - Jul 09)​

Major changes​

Now the following list of traces will be turned on by default:

node-to-client protocol​

• LocalConnectionManagerTracer
• LocalInboundGovernor
• LocalHandshake
• LocalServer

node-to-node protocol​

• Server

See cardano-node#5353. This will be released in node-8.2.0.

We made sure that cardano-client-0.1.0.x library (which is db-sync) is not using the experimental node-to-client protocol. The cardano-client-0.2.0.0 was not affected. See ouroboros-network#4612.

We merged the dynamic block production feature to ouroboros-consensus (also available in the cardano-node-8.2.0 release). Dynamic block production enables hot-swap p2p nodes, which are important for the p2p deployment of block-producing nodes. See ouroboros-network#3159, [ouroboros-consensu#140].

We continued to review the implementation of big ledger peers for eclipse evasion, see ouroboros-network#4462.

Minor changes​

• We refactored the top level cardano-node error handler: cardano-node#5356

Sprint 40 (Jul 09 - Jul 23)​

Major changes​

We integrated ouroboros-network-0.8.2.0 with the master branch of cardano-node for the 8.2.0 release (the version match between ouroboros-network and cardano-node is purely accidental). This includes:

• integration with dynamic block production feature. This feature is documented in the following PR.
• Warm valency for local root peers (see below).

For the full list of features included in the 8.2.0 release from the network side, please take a look at the pre-release nodes.

We improved the memory footprint of peer metrics measured by the P2P stack. Peer metrics are used to decide which peers to demote every churn interval (roughly every hour with some probabilistic fluctuation). See ouroboros-network#4620. The improvement will be available in cardano-node-8.2.0.

We added an optional explicit warm valency to local root peers of the P2P topology file. Previously we used an implicit valency, e.g. the node keeps connections to all of the local roots; with local warm valency the node will pick only that many peers from the local root peers group to connect to. The hot valency (previously known simply as valency) hasn't changed, but it must be smaller than the warm valency as hot peers are selected from warm ones. Warm valency is a useful feature to limit resource consumption if one of the domain names in the local root peer group resolves to many IP addresses. See ouroboros-network#4575, cardano-node#5409.

Network Update​

Key contributions​

We held a series of session to review the implementation of big ledger peers (eclipse evasion). See #4462.

We get a request from a 3rd party to clarify an inconsistency between CDDL spec and protocol implementation. We worked out a nice solution which takes advantage of the any notion available in CDDL. On the Haskell side we provide Any type which gen generate almost any CBOR term (some are excluded only because they are not decoded back to the same form, and we relay on that property). See #4580.

We fixed a bunch of problems of the cardano-cli ping command. It also now has a limited support of node-to-client mini-protocol (the -c option is ignored, as it cannot be supported by node-to-client protocol). Note that the format of messages has changed, timestamps are printed in ISO8601 format. See #4601, #5326, #5313, #30

In order to provide a new flag in the topology file which enables ledger peers when the chain is close to the tip, we continued to work on #4530. This is currently in review, the consensus team will need to provide us with the new api. This feature is useful for two reasons: makes it easier to maintain a topology file, it will also limit the traffic on public roots generated by for example full node wallets and distribute it to ledger peers.

We also continued to work on a blog post which describes the journey of design & implementation of the dynamic P2P network layer. Too be announced soon :).

Other smaller changes​

We limit the concurrency of resolving dns names. Up to 8 root peers or ledger peers DNS names are resolved concurrently, and at most 2 local root peer DNS names. See #4596.

We fixed handshake query timeout in #4603.

We renamed one of the block-fetch decision constructors as requested by the consensus team, see #4608.

Network Update​

Key contributions and advances​

We merged light peer sharing feature, which allows to include inbound peers into outbound governor known peers. This is the primary way for new unregistered nodes to enter the network, which then can be shared using peer sharing. Note that peer sharing is an experimental feature which is disabled until genesis & eclipse evasion as fully implemented. See #3596.

We are making progress reviewing eclipse evasion, #3886.

We fixed another bug in local root peers. We found out that if the local roots where ignored until the first domain name was resolved, see #4583. The bug fix was backported and released in ouroboros-network-0.8.1.1.

We re-started working on dynamically enabling block forging to address issue #3159, which will enable us to release P2P on block producing nodes. See #140.

New cardano-ping / cardano-cli ping release​

We prepared a new release of cardano-ping library which supports the new query feature (query supported versions). See #4589, #4593 and #5313. The new version of cardano-cli ping will use ISO8601 formatted timestamps; also the formatting of ping results is slightly improved, and it will introduce the new --query-versions (-Q) switch. If the remote site supports the query parameter, the command will print:

redacted-ip:port network rtt: 0.064
redacted-ip:port handshake rtt: 0.064010896s
redacted-ip:port Queried versions [NodeToNodeVersionV11 764824073 InitiatorAndResponder,NodeToNodeVersionV10 764824073 InitiatorAndResponder,NodeToNodeVersionV9 764824073 Initiat
orAndResponder,NodeToNodeVersionV8 764824073 InitiatorAndResponder,NodeToNodeVersionV7 764824073 InitiatorAndResponder]

otherwise it will print the negotiation results

redacted-ip:port network rtt: 0.045
redacted-ip:port handshake rtt: 0.101867615s
redacted-ip:port Negotiated version NodeToNodeVersionV10 764824073 InitiatorAndResponder

Note that in that case cardano-cli ping offers InitiatorAndResponder, which allows us to detect whether the remote side is an InitiatorOnly or InitiatorAndResponder. Also cardano-cli ping will no longer announce itself as InitiatorAndResponder, except for the case mentioned above.

Other smaller contributions​

On a request from the Marlow Team, we published haddocks of typed-protocols, which are now available here (#40, #41).

We made a new release of strict-stm-1.1.0.1 on Hackage, which fixed a bug in package description file, #101 .

We also helped to debug a deadlock when using named pipes on Windows in the new RawBearer API. The API is being used to store secret keys only in memory. The PR #4395 is under review.

We also have two more PRs which are under review:

• #4530: enabling ledger peers on a fixed number of slots before the tip of the chain;
• #4580: a PR which fixes inconsistencies in one of our cddl specs.

High level summary​

We continued working on eclipse-evasion. We also analysed and fixed a bug when using DNS names in local root peers. We continued working on engineering blog post about P2P. We released a new version of packages for cardano-node-8.1.0 release.

We improved our CI, removed obsolete scripts add extra validation which checks if CHANGELOG.md files were updated.

We also improved release scripts.

Detailed summary​

• Eclipse evasion: #4462
• Local root peers bug fix: #4559
• Release to CHaP: #4573
• CI improvements: #4572
• Release script improvements: #4573

High level summary​

We started working on a new way to switch between root & ledger peers (see below). We continued to work on eclipse-evasion. We merged changes to Handshake contributed by Galois Inc. We made improvements to our tests (fixed a flaky test, added cddl specs for NodeToNodeVersionData and NodeToClientVersionData). We improved our CI and automated the process of releasing new package version to CHaP.

Detailed summary​

We continued to work on testing eclipse-evasion.

We came up with an idea to limit how full node wallets relay on root peers (currently operated by IOG, in future also CF and Emurgo). We designed a switch to use ledger peers if the node tip is close enough to the current time. For more details see #4530.

We merged changes to the handshake mini-protocol which allow one to query server's node-to-node / node-to-client parameters. We are grateful to Galois Inc. for implementing it, #4256 and #4538. We published new version of packages to CHaP chap-#253.

We added DiffusionError wrapper. Thanks to it, ouroboros-consensus will not duplicate diffusion errors messages in the log, #4537.

We fixed an issue which caused one of our tests to be flaky, #4515.

We added cddl tests for NodeToNodeVersionData and NodeToClientVersionData: #4540, #4544 (in review).

We wrote scripts which will help us release packages as well as verify that we released all the package necessary to build the newest set of packages, #4542.

We renamed the consensus startup tracer and make sure it doesn't log ExitSuccess exceptions, consensus-#71.

We reviewed PR which adds RawBearer API, #4395.

We made series of improvements to our CI:

• #4539: we don't need to install cryptographic libraries in CI;
• #4545: Javier Sagredo (consensus) cleaned up CI after consensus moved to a new repo;
• #4546: we switched to use GitHub merge queues;
• #4549: we made it possible to trigger building haddocks manually;
• #4553: we fixed and enhanced caching of building dependencies.