58 posts tagged with "consensus"

High level summary​

For the past two weeks the Consensus team:

• Resumed work on implementing the initialization and internals of the second iteration of the in-memory ledger database.
• Resumed work on integrating the new ledger database API for UTxO HD.
• Simplified query versioning for downstream users (863 and 403).
• Confirmed the cause of missed leadership checks due to ledger snapshots (868).
• Derived principles for the versioning/declaration of queries, since Conway complicates it (864 and 866).
• Unblocked the Node Team in the work on rollback-insensitive leadership schedule queries (4765).
• Drafted the (Pre-Genesis) Bootstrap State Machine (808).

High level summary​

The Consensus team implemented and tested a patch that does not propagate future headers. It is under review, and we expect it can be released in the next Cardano node version. On the UTxO-HD front, we finished prototyping the LedgerDB and BackingStore redesign, which is required for the LSM-tree integration and might help us implement a more resource efficient in-memory backend. With this prototype finished we can start integrating the rest of the code. We investigated the unexpected performance degradation observed when acquiring the block context. We also released Cardano node 8.7.0 and moved tree-diff outside cardano-ledger libraries.

High level summary​

Over the last two weeks, we managed to run a UTxO-HD capable node in legacy mode, which maintains the baseline memory usage while keeping all the ledger state in memory (as the current node does). This legacy mode could provide an alternative for releasing a UTxO-HD capable node where people can choose whether to store the large part of the UTxO set on disk or not. However, legacy mode involves code duplication, requires several weeks of work before it is production ready (more integration and testing). Moreover, we will not release UTxO-HD before Conway to avoid the risk of delaying the latter. On the other hand, to integrate the LSM-tree backend for UTxO-HD we need to redesign the Consensus storage layer, and this new design might enable us to implement an alternative in-memory backing store that will have a very similar performance and resource requirements as the current Cardano node. Therefore, during the coming months we will focus our efforts on the redesign of the storage layer and the LSM-tree backend.

On the Genesis front, we reviewed the peer simulator for Genesis tests, which was implemented by Tweag and was signed-off by the Consensus team.

We also continued our work on improving the handling of blocks from the future. We presented possible approaches to handling blocks from the future in the Chief Scientist Meeting at IOG, which was very well-received. The discussion with the IOG scientists touched upon the relationship of this approach to Ouroboros Chronos.

Javier Sagredo continues his cycle as release engineer, and he is working on version 8.7 of Cardano node.

High level summary​

This week the Consensus team made progress on two fronts: the question of survivable eclipse duration, which is part of our work supporting Genesis delivery, and how to improve the handling of blocks from the future. Regarding the UTxO-HD branch, we managed to run a node with legacy blocks, which is syncing with mainnet, up to including Alonzo. We also investigated a regression in mempool snapshotting, which was ultimately solved by a Ledger update, and will be fixed in the upcoming Node 8.6 release.

High level summary​

During the past two weeks the Consensus team received additional benchmark results for the UTxO-HD feature that show the resource usage for the in-memory backend is not satisfactory for a mainnet release, and we need to wait on the implementation of a new infrastructure to benchmark the LMDB backend (not likely to happen before next year). While we wait on this, we are evaluating the feasibility of making the UTxO-HD feature switchable, which will enable us to release it as an experimental feature. On the Genesis front we produced the first draft for a Survivable Eclipse Duration Model. We released version 8.5.0 of Cardano node, resumed work on subpar handling of blocks from the future, and improved our tracing system to assist problem troubleshooting in the node.

UTxO-HD​

• The Plutus workload benchmark for the in-memory backend showed no regressions for the metrics of interest, but it does show an increase in resource usage.
• We got additional ad-hoc measurements on memory UTxO-HD consumption. The memory usage of the in-memory backend is not satisfactory for a release. The memory usage of the LMDB backend is considerably lower, but we need to see how much lower we can bring it by running a node whose memory is constrained to 8GB.
• We resumed work on an alternative solution that will make the UTxO-HD switchable. This will enable us to keep the baseline performance by totally disabling UTxO-HD, while allowing users to experiment with the feature if they wish to do so.

Genesis​

• We produced the first draft for a Survivable Eclipse Duration Model (422).

Support​

• Esgen finished his cycle as release engineer. Node 8.5.0 has been released.
• We resumed work on the subpar handling of block from the future (4251).
• We prepared the integration of new tracing events for the next node release. These tracing events will help debugging potential issues in the node (such as the previously mentioned issue).