High level summary​

• Benchmarking: We've performed benchmarks and analyses for Node versions 8.9.2 and 8.10.0.
• Development: Design phase for implementing quick queries in the analysis pipeline has begun.
• Workbench: We're finishing up the new features for the reporting pipeline; Haskell profile definition work is ongoing.
• Tracing: Improving the Prometheus output is ongoing; the node's build info will be accessible as a Prometheus label.
• UTxO Growth: Our tooling has been augmented to support benchmarks starting with a non-empty chain.

Low level overview​

Benchmarking​

We've performed a full set of release benchmarks for Node 8.9.2. Comparing with release 8.9.1, we could not detect any performance risks for that version.

The benchmarks for 8.10.0 have shown a slight improvement in the time the block forging loop needs to evaluate, whilst additionally, resource usage of the cardano-node process was also slightly reduced - a nice performance improvement.

Development​

Our analysis pipeline is based on batch analysis of data from over 50 cluster nodes; it consumes very large amounts of trace output ex post facto, when the actual benchmark has terminated. This is very time-intensive, and not viable for obeserving an additional metric that you later on determine might need consideration.

We're planning to add quick queries into a benchmarking run's trace data to our analysis pipeline. These will be structured such that parameterizable, ad-hoc querying is supported. Initial tests showed that evaluation speed of such queries is fast enough to merit designing a principled, and generalized, syntax for them - and a subsequent implementation.

Workbench​

The reporting pipeline has been augmented with direct support for customizable, and stylable, TeX rendering - currently receiving final touches.

Porting our performance workbench's profile definitions to Haskell, and providing them with an appropriate test suite, is ongoing work. It is our goal to both increase reliable profile definition and validation, and facilitate usage by engineers less familiar with the workbench.

Tracing​

The work to improve system metrics as presented to Prometheus is still ongoing. Type annotations, as well as introducing Prometheus labels for certain metrics to convey (like e.g. build information), will make that interface more versatile. It also facilitates configuration of monitoring or dashboards like Grafana on top of those Prometheus metrics.

UTxO Growth​

For the UTxO scaling benchmarks, we've augmented the workbench with the capability to support injection of a custom synthesized chain into the deployment, and start a benchmark only after replaying that chain - whereas our benchmarks usually start just with a genesis block.

To achieve that, different components of our tooling needed addition of features: distributing that chain to the node cluster, having analysis work without necessarily providing trace evidence of each block in the chain being forged by a benchmarking node. Cluster timing had to be adjusted to account for the gap between genesis start time and the chain tip. However, this entire mechanism opens up the possibility of having a very distinct ledger state at hand for a benchmark - one, that's been particularly crafted via a series of pre-defined transactions constituting the blocks during creation of the synthesized chain.

In the future, we plan to flesh out a more general design of that mechanism, which currently is tied to a very specific use case only.

High-level overview of sprint 58 and sprint 59​

Releases​

We released cardano-node-8.9.1, and we tagged a soon to be released cardano-node-8.9.2 The 8.9.2 release will have a fixed peer sharing support, which we incidentally broke in 8.9.1. We expanded our test suite to cover discovered bugs (see below for more details). Please also see the release tab in our project to see which PRs / issues were included in a given release, the following mapping might also be useful:

• ouroboros-network-0.13.1.0 -> cardano-node-8.9.1
• ouroboros-entwork-0.14.0.0 -> cardano-node-8.9.2

Genesis​

We continued working on network Genesis support:

• ouroboros-network#3396 - churn policy for Genesis;
• ouroboros-network#4813 - outbound governor support for Genesis;
• support for cardano-cli to write a big ledger peers snapshot to disk and for cardano-node to pass it to ouroboros-network.

As well as a feature required by consensus:

• ouroboros-network#4815 - inform consensus about outbound connections state

TxSubmission​

Only a little progress was made due to one of us being on vacation.

Churn and EKG metrics​

While working on ouroboros-network#4815, we addressed technical debt in churn. The PR removes implicit synchronisation (in terms of delays) in favour of explicit synchronisation with the outbound governor. The PR extends EKG counters traced by the node. See below for some graphs.

Documentation​

We updated the documentation on peer sharing, see cardano-node-wiki#44.

Low level details​

Peer Sharing Testing​

We wrote a testing scenario for peer sharing, which simulates a node setup: A -> B <- C, where neither of the nodes B and C have any local roots; they only learn about other nodes through (light) peer sharing. B learns about A and C because they connect to it, while C should learn about A through peer sharing. This test scenario should prevent us from breaking peer sharing in the future in some obvious ways. In the future, we will also work on extending our test suite with peer sharing in mind. See ouroboros-network#4839, ouroboros-network#4841.

EKG / Prometheus Counters​

Note that this is in progress, so some things might still change.

We will provide counters for active (also known as hot) peers, established (e.g. hot & warm) peers and known (e.g. hot, warm and cold) peers. This is the same way one specifies targets in the node's configuration. In addition, the three groups are split into five categories:

• ledger peers
• big ledger peers
• local root peers
• bootstrap peers
• shared peers

In addition, we also provide a counter for root peers, which counts ledger peers, big ledger peers, local roots and bootstrap peers, which correspond to the root peers target TargetNumberOfRootPeers in the configuration.

We also provide counters for ongoing promotions and demotions.

Ledger peers are affected by TargetNumberOf{Known,Established,Active}Peers and TargetNumberOfRootPeers. The gap between TargetNumberOfRootPeers and TargetNumberOfKnownPeers will be filled either with ledger peers or peers, which the node discovered through peer sharing. Big ledger peers are affected by TargetNumberOf{Known,Established,Active}BigLedgerPeers.

Below are some Grafana graphs from an experimental cardano-node branch:

Deprecation policy​

The previous hot, warm / cold EKG counters will also be available, although deprecated after the new ones are released. Sometime in the future they will be removed.

Grafana graphs​

The screenshots are from a node, which is configured without any bootstrap peers, which is why all bootstrap metrics are 0. The node is configured with the following targets:

  "TargetNumberOfRootPeers": 60,
  "TargetNumberOfKnownPeers": 100,
  "TargetNumberOfEstablishedPeers": 40,
  "TargetNumberOfActivePeers": 15,
  "TargetNumberOfKnownBigLedgerPeers": 15,
  "TargetNumberOfEstablishedBigLedgerPeers": 10,
  "TargetNumberOfActiveBigLedgerPeers": 5,

and has a small number of local root peers and one peer in its publicRoots configuration.

Active Peers Metrics​

Established Peers Metrics​

Known Peers Metrics​

Churn Metrics​

High level summary​

• Implemented the new diffusion pipelining criterion.
• Presented, reviewed and merged the March milestone for Genesis.
• Integrated latest changes into Node version 8.10.
• Regarding the UTXO-HD feature we:
  • Completed fixes in test-suites from cardano-node, cardano-api, and ouroboros-consensus. All tests are passing in latest prototype.
  • Prepared UTXO-HD prototype for LMDB benchmarking.
  • Fixed tracing in cardano-node.
  • Started merging code from the UTXO-HD branch into main (1053, 1052, and 1054).
• We're currently working on getting rid of NoThunks errors in Consensus, so that we can enable these tests in CI, for extra assurance.

High level summary​

The DBSync team has released 1 version for mainnet 13.2.0.1 and 3 pre-releases for sanchonet 4.0.0, 4.1.0, 4.2.0. We have continued the integration of Conway (CIP-1694) and the support of off-chain data (CIP-100)

Lower level summary​

• CI build and docker fixes #1670 #1668 #1667 #1662

• Improved committee representation #1662 #1571 #1633

• Offchain metadata partial support #1654

• Refactoring and tech-debt #1635

• Fixing epoch_stake_progress #1620

• Config and modulatirty improvements #1653 #1652

• Updating to node-8.10-pre. Better support for deposits, refunds and proposal state #1673

High level overview​

This week, the Mithril team continued implementing the certification of Cardano transactions in Mithril networks. They worked on scaling the signature and proof generation for mainnet with the compression of the transaction Merkle tree by using sub-Merkle trees of transactions by block ranges. They also made progress in designing low latency certification, investigated a memory leak in the signature/proof process, and worked on retrieving the tip of the chain with the Pallas chain observer. The team almost completed the prototype for decentralizing signer registration with the relay and a peer-to-peer (P2P) network. Additionally, they completed the refactoring of the database providers of the aggregator and re-span the testing-sanchonet network following the release of Cardano node 8.10.0-pre.

Finally, the team made some optimizations on the compilation of their pre-built binaries to fix panics occurring on CPUs without ADX instructions, and created a network configuration file on the repository to facilitate automatic compatibility checks.

Low level overview​

• Completed the issue Create file with Cardano minimum versions in repository #1615
• Completed the issue Deploy testing-mainnet network #1617
• Completed the issue Refactor database module in aggregator #1583
• Completed the issue Client deprecation notice should be written in JSON when --json option is used #1616
• Completed the issue Activate portable feature in mithril-stm by default #1613
• Completed the issue Prepare testing-sanchonet for respin with Cardano 8.10 #1618
• Worked on the issue Store Block Range Merkle roots in signer and aggregator databases #1633
• Worked on the issue Memory leak in Cardano transactions signature/proof #1629
• Worked on the issue Mithril relay broadcasts signer registrations with P2P PubSub #1587
• Worked on the issue ChainObserver supports retrieving the Chain Point of the tip of the chain #1589
• Worked on the issue Add section for manual setup of squid in SPO guide #1610
• Worked on the issue Mithril Signer Local Error Policy : Error 182 - MuxError #1632