46 posts tagged with "network"

High-level overview of sprint 56​

Cardano-Node, Cardano-API, Cardano-CLI​

In the last sprint, the network team was designated to take the Release Engineering Role, as the coming release contains mostly network advances. Armando Santos has been making progress on it. The following packages were published to CHaP:

• ouroboros-network-0.12.0.0, changelog;
• ouroboros-network-protocols-0.8.0.0, changelog;
• ouroboros-network-api-0.7.0.0, changelog;

and other ouroboros-network packages with minor changes. The following packages were also published:

• ouroboros-consensus-0.16.0.0, changelog;
• ouroboros-consensus-cardano-0.14.0.0, changelog;
• cardano-cli-8.20.3.0, changelog;
• cardano-api-8.39.2.0, changelog.

The work on cardano-node is in progress, see cardano-node#5485

We would like to thank Karl Knutsson (CF) for helping us to test the upcoming cardano-node release.

Ouroboros-Network​

We continued working on tx-submission, ouroboros-network#3311

We started to execute an idea to put all configuration values used by the diffusion layer Ouroboros.Network.Diffusion.Configuration module. Currently, default values are spread in different repositories (e.g. cardano-node, ouroboros-consensus). See ouroboros-network#4805.

We restored how syncing works in bootstrap mode to get the same performance characteristics, see ouroboros-network#4809, ouroboros-network#4810.

We started working on Outbound-Governor Genesis mode, see ouroboros-network#4813.

Ouroboros-Consensus​

We requested to modify how bulk-sync mode works to make it use the same mechanism that the newly added LedgerStateJudgement. See ouroboros-consensus#958.

Low-level summary​

We also addressed a minor issue regarding tracing LedgerStateJudgement changes, see ouroboros-network#4816.

High-level overview of sprint 55​

This sprint the networking was nominated for the Cardano Release Engineer role and we've been working on the integration of bootstrap peers with cardano-node. This is a work in progress: cardano-node-5485. cardano-node-8.9 will be the first node which supports the bootstrap peers feature, but we anticipate additional improvements needed before this is used at scale.

We disabled mean reward for new peers in the churn mechanism which was hurting connection in remote destinations. Thanks to Karl Knutsson (CF) for providing evidence and a patch. In future, this mechanism will be further changed, but this requires more analysis. ouroboros-network#4800

We continued working on tx-submission, ouroboros-network#3311

We welcome Marcin Wójtowicz as part of the Networking Team of Cardano Core Tech. He reviewed our documentation as part of his onboarding process: ouroboros-network#4802.

Andrew Westberg (DripDropz) corrected documentation regarding which node-to-node protocol version introduced P2P.

Low-level summary​

• We fixed an issue related to local root peers related to bootstrap peers: ouroboros-network#4799.

Open Source Contributions​

IOSim​

We released io-sim-1.4.1.0 with some new features & improvements:

• QuickCheck monadic combinators: io-sim#140.
• Provide PrimMonad instance: io-sim#141, thanks to Duncan Coutts (Well-Typed).
• Fixed threadDelay in the presence of registerDelays: io-sim#143, thanks to Consensus Team.

High-level overview of sprint 54​

In this sprint, we merged the bootstrap peers (aka weak genesis) changes to the ouroboros-network repository (ouroboros-network#4615). We continued working on integration & testing it using an early development version of cardano-node (cardano-node#5485).

Andrew Westberg (DripDropz) noticed and fixed a mistake in the cddl spec which resulted in misleading documentation; ouroboros-network#4790. The Handshake protocol flag should be called initiatorOnlyDiffusionMode. Library authors providing connectivity with other nodes via the node-to-node protocol are advised to review their implementation.

We continued working on the tx-submission decision logic; ouroboros-network#3311.

Karl Knutsson (CF) implemented:

• a debugging feature which dumps outbound governor state on the SIGUSR1 event (supported on POSIX compatible OSes); ouroboros-network#4730, ouroboros-network#4789.
• limited the rate at which one can discover nodes through peer sharing; ouroboros-network#4778.

Open Source Contributions​

IOSim​

We released a new version of io-sim to Hackage:

• Sebastian Nagel (IOG) fixed a bug in the flushTQueue implementation; io-sim#135
• We fixed the underflow error in threadDelay in the si-timers library (io-sim#129) reported by Nick Frisby (Tweag)

NoThunks​

We released a new version of nothunks to Hackage:

• wherefrom support; nothunks#46
• added Solo instance; nothunks#45
• added instances for Monoid and Semigroup newtype wrappers; nothunks#43

Low-level summary​

• We started using the ImportQualifiedPost extension, hence a large (autogenerated) syntax change; ouroboros-network#4715.
• Code cleanup; ouroboros-network#4792

High level overview of sprint 53​

We integrated bootstrap peers with cardano-node in an experimental branch ([cardano-node-5485], ouroboros-network-4615). We continued working on tx-submission: ouroboros-network-3311.

Karl Knutsson (CF) worked on peer sharing improvements: ouroboros-network-4778.

Erik de Castro Lopo made ouroboros-network packages compile with GHC-9.8: ouroboros-network-4727.

Nick Frisby (Tweag, Consensus Team), modified local-state-query mini-protocol so that one can acquire the immutable tip and thus get information which is not subject to rollbacks: ouroboros-network-4765.

Low level summary​

Other, less significant changes / improvements include:

• we helped diagnose TVar memory leak when using ghc-9.8. This will require more work from us in the near future, see ouroboros-network-4776;
• we fixed a minor issues with block-size in ouroboros-network: ouroboros-network-4775;
• we changed pipelining depth configuration to use Word16 (thanks to Eric de Castro Lopo): ouroboros-network-4774;
• we changed pipelining depth QuickCheck generator: ouroboros-network-4771;
• we moved configureOutboundConnection to ouroboros-network as a private function: ouroboros-network-4773;
• we updated copyright information of network packages: Win32-network-27, ouroboros-network-4761;
• we merged a PR which added a test which could discover the bug fixed in ouroboros-network-4748, see ouroboros-network-4757.

High-level overview of sprint 52​

Happy New Year!

In this short sprint we analysed a failure which happened on a new large cluster that's run by IOG. The process exhausted all file handles and was left without any functional connections. The issues apparently is rare, and thus doesn't impose a high risk.

We also continued working on tx-submission: ouroboros-network-3311.

Detailed description​

It turned out that the process exhausted the number of file handles leaking multiple /proc/{PID}/stat files open. We suspect that the bug is caused by

• using lazy IO in iohk-monitoring-framework, and
• using a recent kernel version

With lazy IO file handles are read as long as the data is required and they are closed only when EOF is reached. We currently suspect that a new linux kernel added something at the end of the /proc/{PID}/stat which is not parsed by iohk-monitoring-framework, so whenever the file is read we leak it (it's never closed) and eventually, there are no file handles to be used by the network layer: the accept loop doesn't return any inbound connection, neither an outbound connection can be created. This issue will be addressed by the profiling team (which owns the logging subsystem).

The fix will be proposed in the future release, in the meantime we suggest to keep observing file handles used by the node.

I would like to thank John Lotoski (IOG), Karl Knutsson (CF), Neil Davies (PNSol) and Michael Karg (IOG) who all contributed to this analysis.

While analysing the log we also found a few smaller issues in the outbound governor which were fixed in [ouroboros-network-#4764].

The IO error indicating exhausting file handles is not currently visible. It is not re-thrown nor logged. This needs to be fixed in a future version. See ouroboros-network-4769.