53 posts tagged with "performance-tracing"

High level summary​

1. Benchmarks for the 1.36 first pre-release bump of the internal components have been delivered, and data shows the component bump is clear for release.
2. SECP benchmarking enablement is underway: the necessary generator features have been implemented, and are now being integrated into the workbench.
3. The new tracing system: in response to the performance regression we previously discovered we are working on pre-planned implementation improvements, and doing more benchmarks.
4. Infrastructure: the Nomad-based workbench backend has been made closer to a cloud deployment scenario. Cleanup in preparation for Cicero CI/CD integration started.
5. Open sourcing: ongoing SRE collaboration on production deployment of performance data publishing.

Performance​

We have ran benchmarks for the first component bump of the upcoming 1.36 release, and we don't see any significant performance changes. The component bumps are therefore clear for release.

Tracing​

For the tracing system regression that we spotted -- even before, we already had plans for further efficiency improvement, and now we are actively pursuing them. The idea is to collect more statically-available information to enable shifting of more tracing decisions from message delivery time to configuration time.

To support this effort, we also started running more benchmarks and enhanced data analysis with relevant metrics.

Infrastructure​

Generation support for Plutus V2 has been implemented and so, with the help of the previously made looped signature-verifying script, the generator is now capable of producing two SECP workloads: verifying either ECDSA or Schnorr signatures. This is now being integrated into the infrastructure -- the generator parametrisation API is being enhanced and the workbench is being extended to handle the new parametrisation.

In addition the workbench is now being enhanced to handle protocol-version-based choices for the Plutus cost model.

The intermediate cloud compatibility iteration of the workbench cloud enablement effort was merged. We are now doing some cleanup work in preparation for starting the Cicero backend, which will bring us nearly completely to the CI/CD integration.

We continue collaboration with SRE on production deployment of data publishing. We now have a gradual rollout plan, which respects the plans for SRE infrastructure feature availability.

We are working on recovering the software dependency manifest feature that was lost with the organisation-wide transition to CHaP.

As usual, a number of smaller workbench, data analysis & reporting improvements have been made.

High level summary​

1. P2P performance investigation is ongoing, in support of the networking team.
2. SECP benchmarking enablement is underway: we already have the script and are working on Plutus V2 generation support.
3. Unexpected setback in the new tracing system: full scale benchmarks have shown a performance regression: local chain syncing benchmarks were an improvement over legacy tracing.
4. On the open sourcing front we added an integrated data dictionary, which is necessary for explaining ourselves to the world. SRE collaboration on production deployment of performance data publishing has started.
5. We have started bringing the Nomad-based workbench backend closer to a cloud deployment scenario.

Performance​

We are supporting the networking team on P2P performance investigation. Generation support for Plutus V2 was started. We have collaborated with the Plutus team to get a SECP benchmark script, which is now ready for use, pending Plutus V2 support. The transaction generator has also been updated to the cardano-api changes.

Tracing​

We ran an initial round of full-scale benchmarks for the new tracing system -- which uncovered a regression relative to legacy tracing, which is contrary to the local chain syncing benchmarks, that showed improvement instead. We added tracing to cardano-tracer, fixing some minor bugs on the way. Network and disk IO metrics are now collected once again and are integrated into analysis.

Infrastructure​

The first iteration of the Nomad-based local workbench backend was completed -- it has reached feature parity with the existing supervisor backend. The next iteration started, bringing it closer to the cloud scenario, by deploying to separate Nomad tasks connected by a virtual network. This will serve as basis for CI and full cloud backends.

We designed and implemented the authoring pipeline for the performance data dictionary, which will be henceforth embedded in our performance reports. We are collaborating with SRE on production deployment of data publishing.

A number of smaller workbench, data analysis & reporting improvements have been made.

High level summary​

On the performance side, the team ran benchmarks for the the P2P feature and the 1.35.4 release. We finished a prototype for performance data publishing. We almost finished the local deployment backend for the workbench using the new SRE deployment infra. We worked on fixing and improving our data analysis pipeline.

On the tracing side, the team worked on isolating a critical issue causing message loss in the remote tracing backend. The issue was resolved and we now have proper end-to-end coverage for the scenario.

Executive summary​

• The new tracing system public release is getting closer, as we're resolving remaining rough edges that are discovered in full-scale deployments. The local benchmarks we ran were already showing improvement relative to legacy tracing, so we expect similar results at full scale.
• The first (local deployment) iteration of benchmarking adopting the new SRE deployment infra is nearly done. We thank Michael Fellinger and Robin Stumm for their assistance. Two further phases remain: CI integration and cloud deployment.
• The benchmarking data publishing prototype is ready. This serves as a springboard for both opening our performance assessment workflow (to support the wider Cardano developer community), and for data provision to the business community. Our next steps are to secure a permanent deployment for this mechanism and to integrate it into the benchmarking infrastructure. This requires collaboration with SRE.