Skip to main content

· 2 min read
Damian Nadales

High level summary​

We continue refactoring the UTxO HD prototype while we wait for the system level benchmarks. We have created a new repository that contains the anti-diff packages used in this prototype.

On the Genesis front, we are preparing another meeting with the researchers to audit the implementation design, and we continued working on basic tests and simplifications.

During the past two weeks we also introduced two new tools. One for dumping CBOR encoded blocks to JSON, and another to serve a local immutable DB.

Workstreams​

UTxO HD Prototype​

We are in the process of refactoring the UTxO HD prototype, while we wait for the system level benchmarks to confirm if the performance of the prototype is satisfactory.

We also set up a repository for the anti-diff package, which required us to refactor the code, write documentation, and prepare a release to CHaP.

Genesis​

We worked on basic tests for the Limit on Eagerness property of Genesis. We also introduced further robustness and simplifications in the Genesis Density governor. Finally, we developed a presentation to engage again with the researchers on our Genesis implementation design.

Technical debt​

Fostering collaboration​

We are in the process of polishing the ouroboros-consensus documentation site, which we will use a the entry point for Consensus related documentation. The first version will not be complete, but we plan on systematically improving it.

Support​

We added a tool to ouroboros-consensus-cardano-tools which allows to dump the Chain DB blocks or any given CBOR encoded blocks as JSON.

We also added another tool that serves an existing immutable DB via BlockFetch and ChainSync. This tool can help in assisting our local benchmarking efforts (for instance Genesis' ChainSync jumping prototype).

· 2 min read
Jordan Millar

2023-02-08 - 2023-02-21

High level summary​

More maintenance and resolving raised issues.

Completed​

docs​

CI & project maintenance​

Developer experience​

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

In Progress​

CI & project mainteance​

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

· 3 min read
Serge Kosyrev

High level summary​

  1. SECP benchmarking: we ran several rounds of SECP benchmarks, refining the benchmark setup as we discovered the properties of the system. After formulating an initial suggested change to the protocol parameters, we're currently running what we consider the final benchmark, to validate the underlying assumptions.
  2. Release benchmarking: we've performed a round of benchmarks for the hotfix 1.35 release update and initiated the 1.35.6 benchmarks.
  3. New tracing: the improvement in the tracing API, with the underlying restructuring, was completed and merged into the node.
  4. New tracing: before going live, we're performing the documentation update, as well as reworking the end user migration guide.
  5. Open sourcing: the benchmarking data publishing has been completed and deployed. After populating it with relevant benchmark data and providing basic user documentation we can go live.
  6. Infrastructure: the cloud workbench backend is progressing well, the networking aspects of multi-region deployment are currently being worked on.
  7. Infrastructure: the NixOps workbench backend is still being worked on, as part of migration from cardano-ops and benchmarking infrastructure unification.

Performance​

We are approaching the end of a chain of SECP benchmarks, as we gradually eliminated deficiencies in the setup as we were discovering them and answering newly appearing questions:

  • we improved the tx/block filling strategy in the generator, to maximise the per-block utilisation of resources and so better approximate the worst-case,
  • after a discovery of what looked like significant per-SC-call overhead, we again tweaked the the tx/block filling strategy,
  • finally, we're redoing all benchmarks together with a value-only run against the backdrop of Mainnet-sized datasets, to balance the suggested adjustment. That also ran into difficulties wrt. limitations of our benchmarking hardware.

In addition, we started benchmarks of the 1.35.6 release.

Tracing​

A rework of the new tracing system's internals and API was merged. It extended the system with introspection, which enabled a range of improvements, some of which were implemented along the way.

Specifically, we were able to completely short-cut processing of messages generated by the tracers that were made provably ineffective by current tracing configuration. Further, now ongoing work enabled by the introspection facilities, includes static validation of documentation and enhanced node state reporting.

Infrastructure​

On the opensourcing/transparency front, the benchmark data publishing machinery was finally fully assembled and put online. As resources permit, we'll work on populating it with benchmarking data, preparing basic documentation and engaging the stakeholders.

The work on the cloud deployment capability of the Nomad workbench backend continued with focus on setting up inter-node networking and removal of locality assumptions. A major step besides those, was completion of a switch-over to the directly-executable derivations, which eliminate the need for creation and distribution of images -- thereby increasing the speed of deployment.

The Nixops workbench backend progressed steadily, reaching minimal deployment capability. The remaining parts are proper shared configuration generation, and porting of the run control functionality from cardano-ops.

· 3 min read
Iñigo Querejeta Azurmendi

High level summary​

The open fronts that the crypto team is working on are:

  • Mithril: We are creating helper functions to single out the usage of unsafe to facilitate auditing. We are also preparing a RFP for an audit of mithril's core library. Exploring future paths of mithril.
  • cardano-base: Decision of whether to continue with BLS12-381 or switch curves. Conversion Praos to PraosBatchCompat ready, as well as KES secure forgetting.
  • KES agent: using snockets and making things testable in IOSim
  • MuSig2: GH actions updated for checking the files whether they end with an empty line. Also, we reorganized the library.

Low level summary​

Mithril​

  • Given that removing the usage of transmute really affects the benchmarks, we decided to group all unsafe functions to facilitate auditing PR#722
  • We have progressed with the RFP document for the mithril-stm library. We are documenting the differences with respect with the original paper.
  • We are exploring possible paths of how mithril could be used 'as-a-service'. Other projects such as sidechains or Catalyst would benefit of such a service. We are at a very early stage of brainstorming how it could work.

cardano-base​

  • There has been a very thorough discussion with potential users of the BLS12-381 bindings if that is the best curve. We have considered alternatives such as Pasta curves, Pluto-Eris or BLS12-377, and considering it's trade-offs. Seems that the most interesting curve to have on main-net is still 381.
  • The team is gaining expertise in SNARKs to be able to experiment with them, and conclude whether the bindings will allow for SNARK verification on main-net in a timely manner.
  • The update VRF PR#341 is finally merged, and we are ready to merge PR#344, which implements conversion functions from the compatible types between Praos and PraosBatchCompat.

KES agent​

  • Use of snockets to send the data directly from the socket to secure memory.
  • We realized that in order for the DirectSerialise / DirectDeserialise classes to work against IOSim, we have to generalize a bunch of additional primitives
  • Our plans are to: (1) Split up MonadSodium into separate typeclasses, each capturing a more sensible concern; (2) Rename those typeclasses to something that reflects their nature better.

MuSig2​

The GitHub Action linelint is used to check the files.

  • A new job for linting is added to the file /.github/workflows/ci.yml. The rules are configured in the file /.github/workflows/.linelint.yml. Some files from the configuration of libsecp were failing, so in the rules in .linelint.yml the failing files are denoted to be ignored by the linter.
  • Folders are reorganized. We created a folder to handle the example. This folder includes the examplemusig2.c, a distinct config.h, and helper.c. The example is enhanced by implementing the functions in the helper for the configuration given in config.h. The number of messages is different than the tests. The example is made more generic to run with a loop.

· One min read
James Chapman

The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

This sprint the team has been preparing several papers for peer review and knowledge sharing and consulting within IO and the community.

Details​

  • R&D Seminar on experience participating in a member based organisation
  • Full day tutorial on performance engineering presented at HiPEAC 2023 conference
  • Finished preparing and submitted a paper on performance engineering to an ACM workshop
  • Preparing a draft paper for future submission about verifying design refinements for distributed system design
  • Consulting on performance design of other IO projects