50 posts tagged with "consensus"

High level summary​

The Consensus team had a very productive meeting with IOG Researchers. We now seem to be in alignment in regards to a strong argument that the Byron and TPraos eras do not need to be checkpointed for an MVP. There is one remaining question (which applies also to the Praos era): how to assess the threat that short forks pose against historical windows that underperformed? We are currently collaborating on that. We also drafted an argument that the updated "Limit on Patience" timeout sufficiently bounds how long the adversary can inflate a victim's overall sync time.

On the UTxO-HD front, the prototype branch was rebased on top of the latest ouroboros-consensus main branch and integrated on top of cardano-node 8.1.1-pre. As a result, the mempool fairness fix that was released recently is now integrated into UTxO-HD. We managed to run a node again with UTxO-HD enabled. We also identified a race condition in the UTxO-HD prototype and fixed it. In addition, we started performing UTxO-HD ad-hoc benchmarks for cardano-node, which uncovered a performance regression on the Network component when using GHC-9.2/9.4. This is being addressed.

Regarding our support activities, we Released fs-sim-0.2.0.0 and are in the process of preparing the 8.2 release of cardano-node. We also identified and started fixing incorrectly-unevaluated thunks in preparation for enabling CI NoThunks tests.

High level summary​

During the Past two weeks we drafted an implementation path for concluding that a node is caught up, which will also be used to back Network's ledger-peer selection (see this issue). We also carried a thorough investigation into the exact feasibility of applying the Genesis rule to certain historical parts of the chain.

On the UTxO-HD front, we are working on improving the ledger tables design and wrapping up the improved DB locking mechanism. We also released packages that are required not only by UTxO-HD but are already used in cardano.

UTxO-HD​

• We have a plan for making the ledger tables in UTxO-HD more ergonomic by mimicking SOP classes like HPure and HAp . In short, we implement generalised versions of important classes like Applicative and Traversable.

Support​

• fs-sim-0.1.0.2 and fs-api-0.1.0.2 were released, which makes them now compatible with GHC up to 9.6.
• ouroboros-consensus-0.7.0.0 was released for cardano-node 8.1, including query serialization fixes for backwards compatibility.

High level summary​

During the past two weeks we brought further improvements into the Genesis design in collaboration with the IO Research and Networking teams. These improvements concern the Genesis selection rule, candidate rule, and root peers usage. See the [Genesis][#genesis] section for more details.

Regarding our UTxO-HD prototype, during the past two weeks we put together a pull-request that improves the DB locking mechanism, started porting the mempool fairness improvements from our main branch, and integrated a new open source library (that implements cancellative monoids) that allowed us to simplify our code and get a small performance gain.

We improved our tooling by releasing an immutable DB server, which can be used for testing and benchmarking purposes, and a db-truncater program, which can be used in disaster recovery and benchmarking scenarios.

Genesis​

The consensus team working on Genesis:

• Improved the genesis selection rule as a result of our interaction with IO Research.
• Studied how the hard-fork combinator handles forecasting at era transitions, and improved our documentation.
• Determined that the simplest candidate rule we had considered will work for the Genesis window at era transitions, at least for the MVP.
• Elaborated concrete proposal for the Genesis State Machine.
• Met with the Networking Team and advised/co-designed how to implement the stop gap usage of public trusted root peers before Genesis is released (it's similar to the Genesis State Machine).

The team is currently re-analyzing the Limit on Patience, which can be less aggressive now that we've re-introduced the Genesis State Machine.

High level summary​

During the past two weeks we made some important progress in the Genesis design. It seems the BlockFetch logic need not be modified for Genesis, although this needs to be confirmed. We started a DoS mitigation handbook and updated our conceptual component diagram to guide the Genesis design. We engaged with the IOG researchers to work on the Limit on Patience attack vector, work in this area is still ongoing. We sketched a design to decouple the CPU load of the node from its responsiveness to the socket. Finally, we discussed with Networking our approach to lower the performance impact of the BlockFetch decision logic, and got green light from them.

We migrated the consensus code to a new repository, splitting it from the ouroboros-network repository, and released version 0.6 of Consensus.

We also merged the mempool fairness improvement to main branch.

Another significant enhancement to our documentation was the addition of an explanation of the hardfork combinator forecast horizon.

See the sections below for more details.

Genesis​

We reviewed the BlockFetch design documentation, and added some source-code comments that emphasize certain properties of the decisions the BlockFetch logic makes that are helping us confirm that Genesis does not require any changes to BlockFetch. We are waiting on input from our former system architect to verify this.

We migrated and updated the conceptual component diagram in the ouroboros-consensus repository which helps us situate the Genesis design and argument.

We engaged with the IOG researchers about the Genesis design. We sketched out a way to address the concern that the Limit on Patiente (LoP) attack vector duty cycle is indeed low, but it's still non-trivial to ultimately conclude it's sufficiently low.

We also sketched a design to decouple the CPU load of the node from its responsiveness to the socket, since the LoP is a relatively tight timeout, and node performance bugs inducing seconds-worth of latency are unfortunately familiar phenomena.

Fostering collaboration​

We added an explanation of a question that we had to explain many times about the exact behavior of the hardfork combinator forecast horizon.

High level summary​

This week the consensus team continued working on the improved DB lock mechanism for UTxO-HD, and modifications to the mempool benchmarks that this prototype requires.

On the Genesis front we validated that the fragment size calculation in BlockFetch is a major performance sink for ChainSync Jumping. By removing it we will get performance that is acceptably close to that of the baseline. We also started investigating a performance fix that does not alter the existing baseline behavior too much. In addition we reviewed our Genesis attack vector calculations.

On the support front we released Consensus 0.4, and we are working on improving our release process, to support the Cardano-wide efforts in this area. We also performed an analysis on the number of file descriptors that consensus use. This information can be used by the node operators to check if the number of file descriptors they want to support are enough.