56 posts tagged with "consensus"

High level summary​

During the past two weeks the team working on the Genesis implementation continued to engage with the researchers, which resulted in various simplifications of the correctness argument for the historical Genesis window. They also decided on an approach for a syncing node to decide that it is (no longer) caught up. This functionality was requested by the networking team.

The team working on the UTxO-HD implementation ran ad-hoc benchmarks that showed performance issues, which are being investigated. They also merged several improvements required for the first UTxO-HD release, and added a package for easing integration with other downstream components.

Regarding our support activities, we integrated the latest Ledger changes into Consensus in preparation for release 8.2 of node.

Genesis​

• We continued to engage with the researchers on our probabilistic model for historical Genesis window, resulting in various simplifications that make the correctness argument more clear while not being excessively conservative.

• We decided on an approach of how to implement functionality requested by the Networking team; namely, how a syncing node can safely conclude that it is (no longer) caught up. Certain parameters are still subject to discussion with the researchers, and we have still have to agree on a concrete API for this functionality with the Networking team.

UTxO-HD​

• We merged the last of the PRs that were part of UTxO-HD improvements for version 0.1: expose UTxO-HD configuration options in the node, refactor ledger tables, and expose a method of computing the UTxO set size.
• We added a new "legacy" cardano block in a new ouroboros-consensus-cardano-legacy-block package that should ease the transition for some downstream packages to UTxO-HD, like db-sync. This is really only useful for downstream packages that use the parts of consensus that don't involve the storage components, in which case we can largely ignore ledger tables. Ignoring ledger tables could also make functionality like block (re-)application more performant for the legacy Cardano block as compared to the actual (UTxO-HD compatible) Cardano block.
• We performed ad-hoc benchmarks of the UTxO-HD implementation, observing a regression in sync speed in the LMDB implementation as well as a regression in memory usage on the in-memory implementation. We are investigating this.

High level summary​

The Consensus team had a very productive meeting with IOG Researchers. We now seem to be in alignment in regards to a strong argument that the Byron and TPraos eras do not need to be checkpointed for an MVP. There is one remaining question (which applies also to the Praos era): how to assess the threat that short forks pose against historical windows that underperformed? We are currently collaborating on that. We also drafted an argument that the updated "Limit on Patience" timeout sufficiently bounds how long the adversary can inflate a victim's overall sync time.

On the UTxO-HD front, the prototype branch was rebased on top of the latest ouroboros-consensus main branch and integrated on top of cardano-node 8.1.1-pre. As a result, the mempool fairness fix that was released recently is now integrated into UTxO-HD. We managed to run a node again with UTxO-HD enabled. We also identified a race condition in the UTxO-HD prototype and fixed it. In addition, we started performing UTxO-HD ad-hoc benchmarks for cardano-node, which uncovered a performance regression on the Network component when using GHC-9.2/9.4. This is being addressed.

Regarding our support activities, we Released fs-sim-0.2.0.0 and are in the process of preparing the 8.2 release of cardano-node. We also identified and started fixing incorrectly-unevaluated thunks in preparation for enabling CI NoThunks tests.

High level summary​

During the Past two weeks we drafted an implementation path for concluding that a node is caught up, which will also be used to back Network's ledger-peer selection (see this issue). We also carried a thorough investigation into the exact feasibility of applying the Genesis rule to certain historical parts of the chain.

On the UTxO-HD front, we are working on improving the ledger tables design and wrapping up the improved DB locking mechanism. We also released packages that are required not only by UTxO-HD but are already used in cardano.

UTxO-HD​

• We have a plan for making the ledger tables in UTxO-HD more ergonomic by mimicking SOP classes like HPure and HAp . In short, we implement generalised versions of important classes like Applicative and Traversable.

Support​

• fs-sim-0.1.0.2 and fs-api-0.1.0.2 were released, which makes them now compatible with GHC up to 9.6.
• ouroboros-consensus-0.7.0.0 was released for cardano-node 8.1, including query serialization fixes for backwards compatibility.

High level summary​

During the past two weeks we brought further improvements into the Genesis design in collaboration with the IO Research and Networking teams. These improvements concern the Genesis selection rule, candidate rule, and root peers usage. See the [Genesis][#genesis] section for more details.

Regarding our UTxO-HD prototype, during the past two weeks we put together a pull-request that improves the DB locking mechanism, started porting the mempool fairness improvements from our main branch, and integrated a new open source library (that implements cancellative monoids) that allowed us to simplify our code and get a small performance gain.

We improved our tooling by releasing an immutable DB server, which can be used for testing and benchmarking purposes, and a db-truncater program, which can be used in disaster recovery and benchmarking scenarios.

Genesis​

The consensus team working on Genesis:

• Improved the genesis selection rule as a result of our interaction with IO Research.
• Studied how the hard-fork combinator handles forecasting at era transitions, and improved our documentation.
• Determined that the simplest candidate rule we had considered will work for the Genesis window at era transitions, at least for the MVP.
• Elaborated concrete proposal for the Genesis State Machine.
• Met with the Networking Team and advised/co-designed how to implement the stop gap usage of public trusted root peers before Genesis is released (it's similar to the Genesis State Machine).

The team is currently re-analyzing the Limit on Patience, which can be less aggressive now that we've re-introduced the Genesis State Machine.

High level summary​

During the past two weeks we made some important progress in the Genesis design. It seems the BlockFetch logic need not be modified for Genesis, although this needs to be confirmed. We started a DoS mitigation handbook and updated our conceptual component diagram to guide the Genesis design. We engaged with the IOG researchers to work on the Limit on Patience attack vector, work in this area is still ongoing. We sketched a design to decouple the CPU load of the node from its responsiveness to the socket. Finally, we discussed with Networking our approach to lower the performance impact of the BlockFetch decision logic, and got green light from them.

We migrated the consensus code to a new repository, splitting it from the ouroboros-network repository, and released version 0.6 of Consensus.

We also merged the mempool fairness improvement to main branch.

Another significant enhancement to our documentation was the addition of an explanation of the hardfork combinator forecast horizon.

See the sections below for more details.

Genesis​

We reviewed the BlockFetch design documentation, and added some source-code comments that emphasize certain properties of the decisions the BlockFetch logic makes that are helping us confirm that Genesis does not require any changes to BlockFetch. We are waiting on input from our former system architect to verify this.

We migrated and updated the conceptual component diagram in the ouroboros-consensus repository which helps us situate the Genesis design and argument.

We engaged with the IOG researchers about the Genesis design. We sketched out a way to address the concern that the Limit on Patiente (LoP) attack vector duty cycle is indeed low, but it's still non-trivial to ultimately conclude it's sufficiently low.

We also sketched a design to decouple the CPU load of the node from its responsiveness to the socket, since the LoP is a relatively tight timeout, and node performance bugs inducing seconds-worth of latency are unfortunately familiar phenomena.

Fostering collaboration​

We added an explanation of a question that we had to explain many times about the exact behavior of the hardfork combinator forecast horizon.