Skip to main content

· 2 min read
Jean-Philippe Raynaud

High level overview​

The Mithril team released a new 2306.0 distribution that implements minor fixes and updates and completes the wiring of the era activation markers reader into the signer and aggregator nodes. They also designed an event store in the aggregator and completed its implementation as a monitoring solution to provide figures for deployment rates of versions on signer nodes before activating an era switch. Additionally, the team created a dedicated command on their nodes’ CLI that allows generating and signing an era marker’s payload to be stored on the Cardano chain, as well as creating dynamic cases for the end-to-end tests they run on the CI.

Low level overview​

  • Released the new distribution 2306.0
  • Completed the epic that implements signer versions deployment monitoring #718:
    • Completed the implementation an event producer/consumer via channel #738
    • Completed the creation of a database and its configuration to save the events on the consumer side #740
    • Completed the creation of events and sending them on the channel on the producer side #741
    • Completed the creation of the signer registration event #742
    • Completed the creation a query to extract the node versions stakes distribution #743
  • Worked on the epic that implements eras behavior switch #707:
    • Completed the loading of era reader adapters from config in the signer and the aggregator #732
    • Completed the implementation of an era cli command in the aggregator #755
    • Completed the implementation of a dynamic matrix of cases in CI end to end tests #760
  • Fixed some bugs:
    • Fixed the unsupported unixepoch() function #757
    • Fixed the problem that prevented some signers from signing on the testing-preview network #730
    • Update SQLite version to 3.40 in aggregator infrastructure #765

· 3 min read
Michael Karg
  • SECP benchmarking: we concluded our benchmarking runs and analyses of the new SECP primitives for the Valentine hard-fork.
  • Release benchmarking: we performed a round of benchmarks for the 1.35.6 release.
  • UTxO-HD benchmarking: we performed first runs for UTxO-HD and are currently refining the benchmarking setup.
  • New tracing: for better accessibility, the new tracing system is being outfitted with introspective capabilities.
  • Infrastructure: with the Nomad cloud workbench backend we were able to perform our first test cluster runs successfully on SRE infrastructure.
  • Infrastructure: the initial NixOps workbench backend has been completed; a PR containing this work, along with many quality-of-life improvements of our tooling, got merged.

Performance

SECP​

  1. For SECP, we settled on a fixed tx count per block, while simultaneously spending as much as possible of the block budget. Thus we were able to minimize the impact of per-SC-call overhead.
  2. The final runs were performed with various fractions, e.g. half, of the current block budget to ascertain how these workloads would fare compared to a value-only run.
  3. The SECP machinery and profiles are currently being generalized into an approach to aim for very specific aspects of a smart contract for benchmarking.

UTxO-HD​

  1. After analyzing initial UTxO-HD runs, it turned out that mempool snapshotting had to be throttled for benchmarking; it affects a lock that UTxO-HD had to introduce into the forging loop.
  2. We're currently adapting the benchmark setup to that, and will then perform a new combination of baseline and UTxO-HD runs.

1.35.6 release​

Benchmarking the 1.35.6 release candidate could attest to a perfectly clean bill of health.

Tracing

Work on the new tracing system's introspective capabilites is ongoing: Immediate use cases of the new API include being able to statically validate generated tracer documentation, as well as providing information of a specific tracing setup in the node via traces themselves. These features will make the new system both more robust, and more accessible.

Infrastructure

Nomad backend​

  1. Work on the cloud deployment capability of the Nomad workbench backend continued; for testing we can automate multiple Nomad clients.
  2. Locality assumptions were removed and job monitoring was refactored.
  3. To facilitate directly-executable derivations, Nomad Job specification files are now self contained with GitHub references and configs needed to run a cluster.
  4. We're currently evaluating different options for genesis distribution in said cluster.

NixOps backend​

The NixOps workbench backend has reached an initial functional stage. Consequently, the relevant PR was merged. It also contained many improvements to our analysis tooling, as well as a structural overhaul of workbench itself. We consider this an important step of future-proofing our benchmarking machinery.

· 3 min read
Damian Nadales

High level summary​

During the past two weeks we got the results from the system level benchmarks for UTxO HD. They showed a substantial performance regression, so we spent some time analyzing the results. We found out the frequency at which ledger snapshots were taken was too high, so we requested the benchmarking team a new run with a more realistic snapshotting policy. We continued refactoring and improving the prototype, and we released UTxO-HD related packages to CHaP.

We met with IOG researchers and networking specialists to discuss the Genesis design, which was well received. We continued working on testing and benchmarking different Genesis prototypes.

We are also working on solving a test failure related to iterators. This work derived in several improvements such as better documentation, a framework for writing unit (and regression) tests, and the possibility of debugging QuickCheck counter examples in the REPL.

Finally, we released ouroboros-consensus 0.2.0.0 and ouroboros-consensus-cardano 0.3.0.0 to CHaP

Workstreams​

UTxO HD Prototype​

We got the results of the first system level benchmarks for UTxO HD. They seemed to indicate a significant regression in performance. After looking into the benchmark logs we found that the benchmark runs took ledger state snapshots too often, due to the default snapshotting policy depending on k, and k being so small in the benchmark runs. Therefore, the next step is to re-run the benchmarks with a snapshotting policy that more closely resembles the one from mainnet.

At the same time, we continued refactoring and cleaning up the prototype.

Also, we prepared the anti-diff packages (fingertree-rm, diff-containers, simple-semigroupoids) and the lmdb related packages (cardano-lmdb and cardano-lmdb-simple) to CHaP.

Genesis​

The Genesis design was presented to the IOG researchers and Peter Thompson from NSol. It was well received. They pointed out one blindspot, but we think it'll be relatively simple to mitigate.

In parallel, we continued developing test and benchmarks for the Genesis prototypes. I particular we tested and implemented a potential fix for increased ChainDB dequeue timings, which partly behaved as we expected, but still needs further investigation. Also we obtained new benchmarking data for the prototype.

Technical debt​

Related to #4183, we developed a DSL for specifying ChainDB unit tests. This will allow us to better understand the counter-examples returned by QuickCheck tests, and to write regression tests for them. Also, we added a module to enable QuickCheck counter-examples to be run on the REPL, allowing for faster debugging feedback. Also, we improved the documentation related to followers (#4372).

We are also working on a design for optimizing the way we handle blocks from the future.

Support​

We released ouroboros-consensus 0.2.0.0 and ouroboros-consensus-cardano 0.3.0.0 to CHaP. Remember that we decided to split the packages related to Consensus into two bundles, one with the core functionality, Cardano-agnostic code, and another bundle with instantiations specific to Cardano.

· 2 min read
Jordan Millar

2023-02-22 - 2023-03-07

High level summary​

General bug fixes

Completed​

docs​

CI & project maintenance​

Developer experience​

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

In Progress​

Documentation​

CI & project mainteance​

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

· One min read
Kostas Dermentzis

High level summary​

The db-sync team created a new tag 13.1.0.2 which is ready to release. We also investigated and had the first working UTxO-HD integration which is one of the potential future risks for db-sync.

Low level summary​

  • Integrated the UTxO-HD feauture branch in kderme/utxo-hd-1. This doesn't use the full on disk storage but keeps things in memory and the plan is to keep it this way for the first iteration. The integration still has some performance issues which we investigate
  • Created tag 13.1.0.2 which upgrades the dependencies of db-sync
  • Fixed an issue related to errors appearing in SMASH #1353
  • Continued with ghc-9.2 integration #1339
  • Worked on an new fixing procedure for #1348. We try to make these procedures work also on older schema version, without the need to migrate to newer schema, which can be very useful for fixing existing snapshots.