82 posts tagged with "consensus"

High level summary​

This week the team working on UTxO-HD discovered a space leak in the peer metrics code. This was communicated to the Networking team who has a proposed fix. The ad-hoc benchmarks that the team ran using a local immutable DB server showed good memory and time performance. We still have to check the performance on a memory constrained machine.

The team working on the Genesis design started onboarding the team of engineers that will implement the new Genesis protocol. This team is also finalizing the statistical model for historical Genesis feasibility.

On the support front, the team drafted an information exchange requirement (IER) for the Networking team to safely and efficiently control peer load.

High level summary​

During the past two weeks the team working on the Genesis implementation continued to engage with the researchers, which resulted in various simplifications of the correctness argument for the historical Genesis window. They also decided on an approach for a syncing node to decide that it is (no longer) caught up. This functionality was requested by the networking team.

The team working on the UTxO-HD implementation ran ad-hoc benchmarks that showed performance issues, which are being investigated. They also merged several improvements required for the first UTxO-HD release, and added a package for easing integration with other downstream components.

Regarding our support activities, we integrated the latest Ledger changes into Consensus in preparation for release 8.2 of node.

Genesis​

• We continued to engage with the researchers on our probabilistic model for historical Genesis window, resulting in various simplifications that make the correctness argument more clear while not being excessively conservative.

• We decided on an approach of how to implement functionality requested by the Networking team; namely, how a syncing node can safely conclude that it is (no longer) caught up. Certain parameters are still subject to discussion with the researchers, and we have still have to agree on a concrete API for this functionality with the Networking team.

UTxO-HD​

• We merged the last of the PRs that were part of UTxO-HD improvements for version 0.1: expose UTxO-HD configuration options in the node, refactor ledger tables, and expose a method of computing the UTxO set size.
• We added a new "legacy" cardano block in a new ouroboros-consensus-cardano-legacy-block package that should ease the transition for some downstream packages to UTxO-HD, like db-sync. This is really only useful for downstream packages that use the parts of consensus that don't involve the storage components, in which case we can largely ignore ledger tables. Ignoring ledger tables could also make functionality like block (re-)application more performant for the legacy Cardano block as compared to the actual (UTxO-HD compatible) Cardano block.
• We performed ad-hoc benchmarks of the UTxO-HD implementation, observing a regression in sync speed in the LMDB implementation as well as a regression in memory usage on the in-memory implementation. We are investigating this.

High level summary​

The Consensus team had a very productive meeting with IOG Researchers. We now seem to be in alignment in regards to a strong argument that the Byron and TPraos eras do not need to be checkpointed for an MVP. There is one remaining question (which applies also to the Praos era): how to assess the threat that short forks pose against historical windows that underperformed? We are currently collaborating on that. We also drafted an argument that the updated "Limit on Patience" timeout sufficiently bounds how long the adversary can inflate a victim's overall sync time.

On the UTxO-HD front, the prototype branch was rebased on top of the latest ouroboros-consensus main branch and integrated on top of cardano-node 8.1.1-pre. As a result, the mempool fairness fix that was released recently is now integrated into UTxO-HD. We managed to run a node again with UTxO-HD enabled. We also identified a race condition in the UTxO-HD prototype and fixed it. In addition, we started performing UTxO-HD ad-hoc benchmarks for cardano-node, which uncovered a performance regression on the Network component when using GHC-9.2/9.4. This is being addressed.

Regarding our support activities, we Released fs-sim-0.2.0.0 and are in the process of preparing the 8.2 release of cardano-node. We also identified and started fixing incorrectly-unevaluated thunks in preparation for enabling CI NoThunks tests.

High level summary​

During the Past two weeks we drafted an implementation path for concluding that a node is caught up, which will also be used to back Network's ledger-peer selection (see this issue). We also carried a thorough investigation into the exact feasibility of applying the Genesis rule to certain historical parts of the chain.

On the UTxO-HD front, we are working on improving the ledger tables design and wrapping up the improved DB locking mechanism. We also released packages that are required not only by UTxO-HD but are already used in cardano.

UTxO-HD​

• We have a plan for making the ledger tables in UTxO-HD more ergonomic by mimicking SOP classes like HPure and HAp . In short, we implement generalised versions of important classes like Applicative and Traversable.

Support​

• fs-sim-0.1.0.2 and fs-api-0.1.0.2 were released, which makes them now compatible with GHC up to 9.6.
• ouroboros-consensus-0.7.0.0 was released for cardano-node 8.1, including query serialization fixes for backwards compatibility.

High level summary​

During the past two weeks we brought further improvements into the Genesis design in collaboration with the IO Research and Networking teams. These improvements concern the Genesis selection rule, candidate rule, and root peers usage. See the [Genesis][#genesis] section for more details.

Regarding our UTxO-HD prototype, during the past two weeks we put together a pull-request that improves the DB locking mechanism, started porting the mempool fairness improvements from our main branch, and integrated a new open source library (that implements cancellative monoids) that allowed us to simplify our code and get a small performance gain.

We improved our tooling by releasing an immutable DB server, which can be used for testing and benchmarking purposes, and a db-truncater program, which can be used in disaster recovery and benchmarking scenarios.

Genesis​

The consensus team working on Genesis:

• Improved the genesis selection rule as a result of our interaction with IO Research.
• Studied how the hard-fork combinator handles forecasting at era transitions, and improved our documentation.
• Determined that the simplest candidate rule we had considered will work for the Genesis window at era transitions, at least for the MVP.
• Elaborated concrete proposal for the Genesis State Machine.
• Met with the Networking Team and advised/co-designed how to implement the stop gap usage of public trusted root peers before Genesis is released (it's similar to the Genesis State Machine).

The team is currently re-analyzing the Limit on Patience, which can be less aggressive now that we've re-introduced the Genesis State Machine.