Skip to main content

82 posts tagged with "consensus"

View All Tags

· 2 min read
Damian Nadales

High level summary​

During the past two weeks we made some important progress in the Genesis design. It seems the BlockFetch logic need not be modified for Genesis, although this needs to be confirmed. We started a DoS mitigation handbook and updated our conceptual component diagram to guide the Genesis design. We engaged with the IOG researchers to work on the Limit on Patience attack vector, work in this area is still ongoing. We sketched a design to decouple the CPU load of the node from its responsiveness to the socket. Finally, we discussed with Networking our approach to lower the performance impact of the BlockFetch decision logic, and got green light from them.

We migrated the consensus code to a new repository, splitting it from the ouroboros-network repository, and released version 0.6 of Consensus.

We also merged the mempool fairness improvement to main branch.

Another significant enhancement to our documentation was the addition of an explanation of the hardfork combinator forecast horizon.

See the sections below for more details.

Genesis​

We reviewed the BlockFetch design documentation, and added some source-code comments that emphasize certain properties of the decisions the BlockFetch logic makes that are helping us confirm that Genesis does not require any changes to BlockFetch. We are waiting on input from our former system architect to verify this.

We migrated and updated the conceptual component diagram in the ouroboros-consensus repository which helps us situate the Genesis design and argument.

We engaged with the IOG researchers about the Genesis design. We sketched out a way to address the concern that the Limit on Patiente (LoP) attack vector duty cycle is indeed low, but it's still non-trivial to ultimately conclude it's sufficiently low.

We also sketched a design to decouple the CPU load of the node from its responsiveness to the socket, since the LoP is a relatively tight timeout, and node performance bugs inducing seconds-worth of latency are unfortunately familiar phenomena.

Fostering collaboration​

We added an explanation of a question that we had to explain many times about the exact behavior of the hardfork combinator forecast horizon.

· One min read
Damian Nadales

High level summary​

This week the consensus team continued working on the improved DB lock mechanism for UTxO-HD, and modifications to the mempool benchmarks that this prototype requires.

On the Genesis front we validated that the fragment size calculation in BlockFetch is a major performance sink for ChainSync Jumping. By removing it we will get performance that is acceptably close to that of the baseline. We also started investigating a performance fix that does not alter the existing baseline behavior too much. In addition we reviewed our Genesis attack vector calculations.

On the support front we released Consensus 0.4, and we are working on improving our release process, to support the Cardano-wide efforts in this area. We also performed an analysis on the number of file descriptors that consensus use. This information can be used by the node operators to check if the number of file descriptors they want to support are enough.

· One min read
Damian Nadales

High level summary​

This week the consensus team finished the UTxO-HD prototype refactoring. We are now working on improving the DB lock mechanism to improve performance. We also introduced several improvements to the file system abstraction and simulation layer (fs-sim), which culminated in the release of fs-sim-0.1.0.0 and fs-api-0.1.0.0 to CHaP.

On the Genesis front we distributed the updated Genesis design document, soliciting feedback from Networking Team and IOG Researchers. We also opened up a PR for the adversarial leader schedule QuickCheck generator, which is being reviewed.

On the support front, we got a new Consensus version that can use different fundamental VRF crypto primitives for Babbage and Conway eras.

On the tech debt front we fixed an bug in the followers logic, which was discovered by our QuickCheck property tests.

· One min read
Damian Nadales

High level summary​

This week the consensus team continued working on the refactoring of the UTxO HD prototype, and design and testing of Genesis. We also extracted the fs-sim package, which provides a file-system abstraction layer that can be used for testing and simulation. This makes the Consensus code base smaller, while providing a package that the community can reuse and contribute to. We also fixed a failing property test related to iterators. We are also working on mempool and VRF improvements.

Low-level details​

· 3 min read
Damian Nadales

High level summary​

The Consensus team continued working on refactoring and improving the UTxO-HD prototype, and introducing improvements to the lmdb related packages. In particular we identified an opportunity to gain performance by handling locks in a more optimal way.

On the Genesis front, we sketched a mitigation for an issue that PNSol and Researchers caught. We also came op with a road map for not only testing the Genesis prototypes, but also for enriching the tests we already have.

Regarding technical debt, next to some minor improvements, we created component-level micro-benchmarks for adding transactions to the mempool. The results of these benchmarks will be published in the ouroboros-consensus web page.

We also finished moving the Consensus documentation to the ouroboros-consensus repository, released ouroboros-consensus 0.3.0.0, and reduced the time GitHub actions take in ouroboros-network.

Workstreams​

UTxO HD Prototype​

We continued working on refactoring and improving the UTxO-HD prototype. As a result of the first round of sytem-level benchmarks, we identified an opportunity to optimise the way we handle locks to improve performance (#4393).

Also, we introduced several improvements to the lmdb related packages:

Genesis​

We sketched out a mitigation of the issue that PNSol and Researchers caught in the Genesis design.

We came up with a road map for testing the Genesis prototypes, including early milestones that are applicable to today's master branch, ie tests that are useful before Genesis, and that will be nicely enriched when we do add Genesis.

We developed the aforementioned tests, specifically a QuickCheck generator for the Honest leader schedule and one as-aggressive-as-possible Adversarial leader schedule that together satisfy the Praos properties that the Consensus design takes as invariants.

We investigated why the improved ChainDB queueing implementation behaves differently in the baseline compared to the prototype, and we are close to having a full picture of how the Consensus components interact during bulk sync.

Technical debt​

We created component-level micro-benchmarks for adding transactions to the mempool. We plan on extending this to more mempool actions and different types of blocks. We store the benchmark data to make it available to the GitHub action that publishes the benchmarks results.

Other minor improvements include:

  • Removal of Test.Util.Classify in favour of Test.StateMachine.Labelling.
  • Addition of -Wunused-packages to the default ghc-options for Consensus packages.

Fostering collaboration​

We finished moving the Consensus documentation from ouroboros-network to ouroboros-consensus, in preparation for migrating the code to the latter repository.

Support​

We released ouroboros-consensus 0.3.0.0.

We reduced the load in the ouroboros-network GitHub actions, thereby reducing the time CI jobs take.