91 posts tagged with "consensus"

High level summary​

• Reviewed the UTxO HD PR, and started addressing Review comments.
• Engaged with Researchers about to discuss the HFC simplification proposal.
• Reverted the Babbage->Conway era transition workaround, clarifying the semantics around stake from pointer addresses (see #1297).
• Well-Typed worked on two features for lsm-tree:
  • snapshots (for persisting ledger snapshots)
  • table union (for storing more parts of the ledger state on disk)
• Addressed minor tech debt issues (#1269).

High level summary​

• Investigated performance improvements in mempool snapshotting in recent node benchmarks and discussed potential further improvements.
• Started the review of the UTXO-HD feature branch after all the issues have been resolved.
• Published io-classes-extra, which hosts concurrency utilities that were extracted from the consensus repository.
• Elaborated the plan for the last quarter of 2024. You can reach out to our Discord channel for any comments or suggestions.
• In the context of UTXO-HD, Well-typed presented another LSM-tree milestone. The implementation includes incremental merges, which prevents substantial spikes in resource usage (CPU, disk, memory), and duplicating table handles, which is crucial for efficiently representing sequences of ledger states. The test coverage of the LSM-tree library was improved as well.

High level summary​

• Released diff-containers-1.3.0.0, fingertree-rm-1.0.0.3, cardano-lmdb-0.4.0.2, and cardano-lmdb-simple-0.7.0.1.
• Improved IOSim:
  • Added MonadLabelledMVar (176).
  • Improved the Show instance of ScheduleMod (175).
  • Ensure labels are printed where possible in IOSimPOR (174).
• Released Consensus packages for Node 10.0.
• Simplified Cardano hard fork triggers (1282.
• Decreased max permissible clock skew for headers from the future from 5s to 2s (1279).

High level summary​

Over the past two weeks, we worked on establishing a first baseline for the Consensus QTA (#1256). This work is also helping inform discussions with the Networking team about sync performance goals in general and for specific improvements, such as Genesis.

UTXO-HD was rebased atop main and the tests are passing.

Finally, the team has also worked to improve the Consensus layer's robustness and security.

High level summary​

• Debunked our working theory on the cause of performance degradation when taking a ledger snapshot. We are now back to the UTXO set as the first contributing cause to said degradation, and together with the Ledger team we have proposed a way decrease the number of allocations when serializing the ledger state.
• Developed the first and second draft scripts for estimating the bandwidth necessary to ensure the CPU is the bottleneck when syncing (#1240). This is informing us and the Networking Team how to refine BlockFetch for the syncing node (especially for Genesis).
• On the UTXO-HD front:
  • After addressing several issues found during benchmarking and testing, the performance team ran benchmarks on the utxo-hd-9.1 branch, yielding positive results. The nodes function without errors. The memory and CPU usage is almost on par with the 9.1 node.
  • A tool has been provided to convert ledger state snapshots from pre-UTxO-HD nodes to UTxO-HD nodes, allowing users to use UTxO-HD right away without needing to replay the chain (since they can use their locally stored ledger state after converting it with the aforementioned tool).
  • The SDET team will run integration tests on the utxo-hd-9.1 branch. If the tests pass, we will start working on wrapping up the documentation and preparing the branch for merging once it is decided to release this feature.
  • Bear in mind that:
    • This UTxO-HD release uses an LMDB backend (but it also provides an in-memory backend). The LSM-tree backend should arrive Q1 2025.
    • UTxO-HD is just the first step of a bigger initiative for moving parts of the ledger state to the disk storage, lowering the memory requirements of the node and contributing to long term sustainability of Cardano.