High-level overview of sprint 43​

In this sprint, we received contributions from CF & Galois. Karl Knutsson (CF) has addressed various issues regarding peer churning in P2P, timeouts and our WireShark dissector. While the Galois developers focused on addressing issues from their review last year. See below for more details.

We continued working on bootstrap peers ouroboros-network-#4661.

We refactored our test suites: they are split into io-tests which require to be run natively on all platforms (these tests mostly contain tests that require IO system calls) and sim-tests which are platform independent. We run io-tests on all supported platforms (e.g. x86_64-linux, x86-64-darwin, aarch64-darwin and x86_64-w64-mingw32 (Windows)) natively. The sim-tests are not executed on Windows due to memory limitations on GitHub Actions runners. ouroboros-network-#4653

We also started rebasing typed-protocols refactoring branches.

Marcin was appointed as the cardano-node release engineer for the 8.4.0-pre version. So far he integrated cardano-ledger-conway-1.8 and ouroboros-network-0.9.1.0 to ouroboros-consensus, cardano-cli and cardano-api. Once we will have an integration branch for cardano-node, cardano-ledger-conway-1.8 and ouroboros-consensus packages can be released to CHaP and PRs can be merged once they go through review & CI.

We also fixed some smaller issues regarding peer sharing (both were discovered by Karl from CF). More details are included below.

Progress on P2P addoption​

SPO relays​

There are currently ~2000 relays running P2P enabled nodes that belong to 557 pools with a combined stake of 7900Mil Ada. On 16th of August it was ~1700 relays, 531 pools with a combined stake of 7700Mil Ada.

The following graphs show several different versions of relays running on the mainnet. The green line NodeToNodeVersionV10.True denotes P2P relays, which slowly increase over time. The V9 and earlier versions of the node-to-node the protocol indicates nodes version 1.35.x or earlier.

Data has been kindly provided by CF and their mainnet monitoring infrastructure.

IOG relays​

As of this week, 90% of IOG relays are running a P2P setup. In the next sprint all IOG relays will be running P2P.

Detailed description​

In this sprint, we got a few contributions from CF:

• Karl made peer churning mechanism less aggressive ouroboros-network-#4656; and
• he added timeouts for idle states in ChainSync & KeepAlive miniprotocols. These timeouts help a node remove idle connections from the responder (server) side ouroboros-network-#4648.
• he improved the WireShark dissector by adding support for the peer-sharing mini-protocol ouroboros-network-#4656.

Galois has been making progress in addressing some of the issues they raised in their review (last year):

• Mark Tullsen (Galois) has been working on refactoring the main entry function of P2P ouroboros-network-#4661 (in review);
• Sam Cowger (Galois) has been reviewing all use cases of error and unsafe calls ouroboros-network-#4661 (in review).

Peer Sharing​

• Light peer sharing is only enabled when peer sharing is turned on ouroboros-network-#4652;
• Handshake incorrectly reports peer sharing value. It's supposed to relay the remote value, but instead, it returns the local value. ouroboros-network-#4642 (in review).

Async Demotion Test Fix​

• We fixed an async demotion test failure which turned out to be a weakness of the test itself rather than a bug in the connection manager. ouroboros-network-#4655

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent updates or improvements include:

• Work on cardano-parts repo continues and was utilized by the sanchonet-demo repository for demonstration purposes
• Work on a new capkgs repository started -- "Content Addressed Packages"
  • This repository provides common Cardano release and pre-release binaries in a pure content addressed nix package type via flake output
  • These packages don't have the associated IFD, long eval times and large source dependencies that the originating source repositories often do
  • Consumers of such packages, such as devShells or nixos machines, can benefit from the improved performance this offers

Lower level summary​

Capkgs​

• Content addressed packages -- new repo: capkgs

Cardano-ops​

• Cardano p2p relay updates, dbSync snapshots server update, monitoring adjustments: cardano-ops-compare

Cardano-parts​

• Flake devShell module improvements: cardano-parts-pull-4
• Cardano-world entrypoints migrated to flake parts module: cardano-parts-pull-5
• Cardano-world jobs migrated to flake parts module: cardano-parts-pull-6
• Drep delegation fixups: cardano-parts-pull-7
• Utilize content addressed binaries, WIP: cardano-parts-pull-8
• Cardano-node module preparation branch, WIP: cardano-parts-compare

Cardano-perf​

• Availability zone pinning and explorer mods: cardano-perf-compare

Cardano-world​

• Remove performance nodes for migration to cardano-perf: cardano-world-commit

Sanchonet-demo​

• Sanchonet demonstration repository: sanchonet-demo

2023-08-16 - 2023-08-29

High level summary​

CLI is making progress towards phases 3 and 4 of SanchoNet.

• Ability to create info, no-confidence and new committee governance actions.

• Voting as DREP is possible now on the CLI

• API is now using Ledger's PParams and VotingProcedures

• Expose following queries from consensus:

  - GetGovState
  - GetDRepState
  - GetDRepStakeDistr
  - GetCommitteeState
  - GetConstitution
  

cardano-cli​

• Add goverance info action creation to era based cli
• Add create-no-confidence command to era based cardano-cli
• Add create-new-committee command to era based cli
• Enable --drep-script-hash option
• Remove experimental subcommand
• Upgrade to cardano-api-8.17.0.0
• New governance drep id command
• Add governance query commands
• Add --verification-key option to committee key-hash command
• Read and write VotingProcedures files instead of VotingEntry files
• Release 8.6.0.0
• Merge LegacyClientCmdError into CmdError
• Add test for conway governance action create-constitution
• Move legacy errors out of legacy command structure modules
• Governance drep and vote command groups
• Move command errors from era-based to CmdError module
• Move some error types
• Delete unused module
• Refactor: Use maybeFeatureInEra instead of featureInEra where possible
• Remove all uses of toS, purer, cborError and intercalate from Cardano.Prelude
• Remove example era-based parsers we don't need anymore
• Bump CHaP index in preparation for 8.3.0 pre

cardano-api​

• New version cardano-api-8.17.0.0
• Replace ProtocolParameters usage with ledger's PParams
• Use injective type families to improve type inference
• Delete TxVotes and VotingEntry and use VotingProcedures instead
• New functions emptyVotingProcedures, singletonVotingProcedures and mergeVotingProcedures
• New makeStakeAddressAndDRepDelegationCertificate function
• Expose ledger lenses for governance types
• New version cardano-api-8.16.1.0
• Add VotingProcedures type
• Release 8.16.0.0
• Update createAnchor to use hashAnchorData
• Release 8.15.0.0
• Add consensus governance queries

cardano-node​

cardano-testnet​

docs​

CI & project maintenance​

High level overview​

The Mithril team created an explorer page with the registered signers to give recognition to the SPOs already running on mainnet. They also worked on refactoring and standardizing errors in the Mithril nodes, and they enhanced the clarity and reduced verbosity of the logs for the signer and aggregator nodes. Additionally, they started working on the implementation of zstandard compression of the snapshot archive, and streaming its download and decompression in the client.

Finally, they implemented machine readable progress information in the client, and streamlined the process of verifying the registration status of a signer.

Low level overview​

• Completed the issue Create explorer page with registered signers #1097
• Completed the issue Simplify logs on the Signer/Aggregator #1106
• Completed the issue Refactor (de)serialization of crypto entities #668
• Completed the issue Mithril client last snapshot download #1078
• Completed the issue Use the '/signers/registered' route to verify if a signer is registered #1152
• Worked on the issue Provide progress information with '--json' option in Client #1095
• Worked on the issue Use 'zstandard' compression for snapshot archives #876
• Worked on the issue Make client download and extract the archive simultaneously #1115

High-level summary​

This week, the Hydra team conducted the monthly review meeting for Agust,which continously ensures transparent communication and project evaluation.

In terms of community engagement, the Hydra team gave a Hydra master-class at RareEvo workshop as an educational initiative for the community. Additionally, the team updated the Hydra demo and tutorial to include Mithril and latest release changes.

They also focused on exploring the Hydra network resiliance in order to improve hydra-node availability, as it becomes stuck very easily and that leads to a bad user experience.

What did the team achieve this week​

• Hydra master-class at RareEvo workshop!
• Monthly report & review meeting
• Updated Hydra tutorial including Mithril and re-record demo/tutorial stuff
• Groomed network resilience and explorer features #188
• New ADR for “Resourced based API” #1028

What are the goals of next week​

• Have a poc on network resilience working #188
• Fix issue #931: “Chain state in head state not updated on replayed observation”
• Fix issue #1039: "Close transaction dropped from cardano-node"