High level summary​

During the past two weeks we made some important progress in the Genesis design. It seems the BlockFetch logic need not be modified for Genesis, although this needs to be confirmed. We started a DoS mitigation handbook and updated our conceptual component diagram to guide the Genesis design. We engaged with the IOG researchers to work on the Limit on Patience attack vector, work in this area is still ongoing. We sketched a design to decouple the CPU load of the node from its responsiveness to the socket. Finally, we discussed with Networking our approach to lower the performance impact of the BlockFetch decision logic, and got green light from them.

We migrated the consensus code to a new repository, splitting it from the ouroboros-network repository, and released version 0.6 of Consensus.

We also merged the mempool fairness improvement to main branch.

Another significant enhancement to our documentation was the addition of an explanation of the hardfork combinator forecast horizon.

See the sections below for more details.

Genesis​

We reviewed the BlockFetch design documentation, and added some source-code comments that emphasize certain properties of the decisions the BlockFetch logic makes that are helping us confirm that Genesis does not require any changes to BlockFetch. We are waiting on input from our former system architect to verify this.

We migrated and updated the conceptual component diagram in the ouroboros-consensus repository which helps us situate the Genesis design and argument.

We engaged with the IOG researchers about the Genesis design. We sketched out a way to address the concern that the Limit on Patiente (LoP) attack vector duty cycle is indeed low, but it's still non-trivial to ultimately conclude it's sufficiently low.

We also sketched a design to decouple the CPU load of the node from its responsiveness to the socket, since the LoP is a relatively tight timeout, and node performance bugs inducing seconds-worth of latency are unfortunately familiar phenomena.

Fostering collaboration​

We added an explanation of a question that we had to explain many times about the exact behavior of the hardfork combinator forecast horizon.

• Benchmarking: The benchmarks and performance investigations for the new 8.0 release branch are ongoing.
• New tracing: Performance optimization of the new tracing system is paying off and we could notably shrink its resource footprint.
• Analysis pipeline: An exhaustive documentation and dataflow diagram for our analyses is being worked on.
• Infrastructure: The plutus-apps flake input for cardano-node has finally been removed.
• Nomad backend: A PR implementing placement of benchmarking clusters has been merged.

Benchmarking

The performance investigations on the 8.0 release branch have lead to pinpointing and addressing incosistent behaviour. For that, we created yet another local reproduction with the workbench's forge-stress benchmark.

Currently we're working on scaling up the dataset size (UTxO and delegations) on the AWS cluster to gain further insight into 8.0 and subsequent releases.

Additionally, we've refined the trace-bench family of profiles that target benchmarking our own new tracing system.

Tracing

Optimization of the tracing system has identified several locations where inefficient serializations were used; those were not originally intended to run on a performance-critical codepath. We've worked on improving those, as well as eliminating cases of redundant conversion between different serialization formats. This has brought down both memory and CPU impact of the tracing system.

Infrastructure & Analysis

Dataflow documentation​

The LogObject CLI locli is at the heart of our analysis and reporting pipeline. To increase its accessibility and facilitate further development, we're creating a detailed and illustrated documentation of all dataflows that happen during analysis and reporting.

Remove redundant Plutus flake input​

This step is the conclusion of porting Plutus benchmarking scripts to our own library. By finally removing the now unnecessary flake input, we simplify the dependency graph for cardano-node, as well as enable immediate feedback when developing Plutus benchmarks.

Nomad backend​

Sophisticated placement of nodes across various regions of the globe is a cornerstone of the model cluster we use for benchmarking. This capability has now been added to the Nomad backend and can be controlled with Nomad job descriptions. A PR with this, along with various quality-of-life improvements, has been merged to master.

High level summary​

Lower level summary​

• Prepared a schema change which is expected to speed up queries and help clients #1389.
• Ran a big number of benchmarks on the above to evaluate if this change is reasonable.
• Performance optimization related to rewards #1382
• Using the cache in more places to reduce recourses usage #1380
• Evaluated risks related to UTXO-HD integration
• Worked on a feauture request to split the tx_out to an address table #1396
• Worked on another feauture request to prune the consumed entries from the tx_out table #1398
• Prepared for CIP-1694 integration

High level summary​

The open fronts that the crypto team is working on are:

• cardano-base: New KES structure finally merged. Included additional tests to BLS, and including test vectors created with zkcrypto's library.
• KES agent: Reaching state of MVP. Rewriting KES agent networking code to use the Snockets abstraction.
• Sidechains: Started implementing ECC chip of JubJub over BLS12-381.

Low level summary​

cardano-base​

• #255 finally merged 🎉
• Additional tests for BLS bindings #384
• Open PR for including test vectors created with zkscrypto's library, PR#397

KES agent​

• Introducing the SignKeyWithPeriodKES wrapper type, which bundles a KES key with its corresponding KES period PR#389.
• Introducing the DirectSerialise API into cardano-base, which allows us to write mlocked memory directly to a file descriptor, without going through intermediate encodings stored on the GHC heap (which is a no-no due to secure forgetting requirements) #317.
• Figuring out how to generalize the KES, DSIGN, and mlocking code in cardano-base such that it can run in IOSim
• Rewriting KES agent networking code to use the Snockets abstraction, which gives us a platform-independent sockets layer (i.e., it will also work on Windows), and also allows us to run networked code in IOSim for testing purposes
• Refactor MonadSodium into MonadMLock PR#388.

Sidechains​

• Started working on ECC chip for JubJub over BLS (problems still with the multiplication constraint)
• Use macros used by PSE library to try and merge our fork of halo2curves into PSE's repo

The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

The team is working on porting an existing formalisation of Ouroboros Praos to a newer foundation.

Details​

• The team worked on documentation.

• The team worked on an application layer for the Praos formalisation.

• The team is hiring a performance engineering intern - https://apply.workable.com/io-global/j/BFEC5B0AFF/