Skip to main content

Performance & tracing update

· 3 min read
Michael Karg
  • Benchmarking: We performed a series of benchmarks aimed at the new 8.0 release branch and built a timeline from the 1.35 releases to that branch.
  • New tracing: Work on safeguarding the new tracing system performance-wise is ongoing. A practical use case for data points is being tackled with Galois.
  • Analysis pipeline: We're working on automatically obtaining a detailed manifest for each run.
  • Infrastructure: The library for benchmarking Plutus scripts has been merged. Also, we've laid the ground for including GHC profiling data into our workbench.
  • Nomad backend: The first iteration of a distributed / multi-client Nomad cluster has been merged.

Benchmarking

We have performed various cluster runs targeting the 8.0 release branch. That way we were able to catch an inconsistency in behaviour early on. This led to the creation of a specialized workbench profile epoch-transition for local reproduction of what we observed on the benchmarking cluster.

Furthermore, we bridged the gap between the run data from the 1.35.x releases to the the new 8.0.x release branch. This included walking the master branch backwards and pinpointing the order, as well as the dates and commits of all relevant component bumps. This timeline is absolutely crucial in locating possible regressions for the new release branch, as it provides the exact points in history we would need to target with a comprehensive set of benchmarks.

Tracing

In-depth performance analysis of the new tracing system has already yielded results and helped us smoothing some rough edges. However, this work is still ongoing.

In coordination with Galois, who are developing a system assurance service by observing a number of cardano-nodes, we're working with the implementation of data points which the node provides during runtime. While making the view on data points expressive enough for the external service, the computational burden inside the node needs to be kept to an absolute minimum. We're currently in ideation about whether cardano-tracer could be extended with a richer feature set to that end.

Infrastructure & Analysis

Detailed manifest​

A run manifest documents, among other things, the component dependencies that were used for a specific build the run has been performed with. These dependencies come from different package sources, have different versioning policies, and an identical package version might provide different performance characteristics depending on the exact commit used for the build. This manifest will greatly increase insight into where changes in measured behaviour might have originated by making all component bumps visible and accessible.

GHC profiling inside workbench​

The workbench has been equipped with a new -profnix profile flavour. This enforces a -fprof-auto build for all node-related packages. The type of profiling data generated by the GHC runtime can be customized and will enter statistical analysis. The relevant PR for this new feature has already been merged to master.

Nomad backend​

The added feature for a multi-client Nomad cluster greatly enhances how jobs are organized by the backend and mapped within specific instances. This results in great maintainability while not giving up on flexibility. However, work on that feature is still ongoing.