- Benchmarking: We performed a series of benchmarks aimed at the new
8.0 release branch and built a timeline from the 1.35 releases to that branch.
- New tracing: Work on safeguarding the new tracing system performance-wise is ongoing. A practical use case for data points is being tackled with Galois.
- Analysis pipeline: We're working on automatically obtaining a detailed manifest for each run.
- Infrastructure: The library for benchmarking Plutus scripts has been merged. Also, we've laid the ground for including GHC profiling data into our workbench.
- Nomad backend: The first iteration of a distributed / multi-client Nomad cluster has been merged.
Benchmarking
We have performed various cluster runs targeting the 8.0 release branch. That way we were able to catch an inconsistency in behaviour early on. This led to the creation of a specialized workbench profile epoch-transition for local reproduction of what we observed on the benchmarking cluster.
Furthermore, we bridged the gap between the run data from the 1.35.x releases to the the new 8.0.x release branch. This included walking the
master branch backwards and pinpointing the order, as well as the dates and commits of all relevant component bumps. This timeline is absolutely crucial
in locating possible regressions for the new release branch, as it provides the exact points in history we would need to target with a comprehensive set of
benchmarks.
Tracing
In-depth performance analysis of the new tracing system has already yielded results and helped us smoothing some rough edges. However, this work is still
ongoing.
In coordination with Galois, who are developing a system assurance service by observing a number of cardano-nodes, we're working with the implementation
of data points which the node provides during runtime. While making the view on data points expressive enough for the external service, the computational
burden inside the node needs to be kept to an absolute minimum. We're currently in ideation about whether cardano-tracer could be extended with a richer
feature set to that end.
Infrastructure & Analysis
Detailed manifest
A run manifest documents, among other things, the component dependencies that were used for a specific build the run has been performed with.
These dependencies come from different package sources, have different versioning policies, and an identical package version might provide different
performance characteristics depending on the exact commit used for the build. This manifest will greatly increase insight into where changes in measured behaviour might have originated
by making all component bumps visible and accessible.
GHC profiling inside workbench
The workbench has been equipped with a new -profnix profile flavour. This enforces a -fprof-auto build for all node-related packages. The type of
profiling data generated by the GHC runtime can be customized and will enter statistical analysis. The relevant PR for this new feature has already
been merged to master.
Nomad backend
The added feature for a multi-client Nomad cluster greatly enhances how jobs are organized by the backend and mapped within specific instances.
This results in great maintainability while not giving up on flexibility. However, work on that feature is still ongoing.
