High level overview​

The Mithril team completed the Mithril protocol’s mainnet beta launch: the release-mainnet network is now consistently producing new certificates and snapshots! The team has released a new distribution 2331.1 which includes some optimizations and bug fixes: the bootstrap time of a Cardano node restored from a Mithril snapshot has been significantly reduced, and a bug that made the aggregator produce corrupted archives has been fixed. They also kept working on the implementation of the stress test tool for benchmarking the aggregator: the first phase has been implemented, as well as a monitoring feature. Additionally, they kept implementing the refactoring of the serialization/deserialization of the entities of the cryptographic library, and working on the post deployment of the mainnet infrastructure.

Finally, they completed the rewriting and enhancements of the documentation website, and they completed the runbook for operating a production Mithril network.

Low level overview​

• Released the new distribution 2331.1
• Completed the epic Release 'mainnet' Mithril network #918
• Worked on the epic Benchmark performances of Mithril Aggregator #904:
  • Completed the issue Design & implement basic stress test tool for aggregator - phase 1 #991
• Worked on the epic Post deployment 'mainnet' infrastructure #1091:
  • Completed the issue Create Production Infrastructure Runbook #1085
  • Completed the issue Ingest 'mainnet' metrics/logs in Grafana #1122
• Worked on optimizations:
  • Completed the issue Add a 'clean' file to extracted database in client #1131
  • Worked on the issue Create explorer page with registered signers #1097
• Worked on refactoring:
  • Worked on the issue Refactor (de)serialization of crypto entities #668
• Worked on bugs:
  • Completed the issue Client fails to unpack some snapshot archive #1137
  • Completed the issue The 'release-preprod' network stopped producing certificates #1114
  • Completed the issue The 'release-mainnet' network does not show up in the explorer #1111
• Worked on dependencies:
  • Completed the issue Upgrade Cardano node to '8.1.2' #1090
• Worked on troubleshooting:
  • Completed the issue Signer can't sign with 'Invalid signature for party' error #1125

High level summary​

This week the Consensus team worked on integrating the latest node release into the UTxO HD branch. Our ad-hoc benchmarks in which we sync a node from scratch showed that the LMDB backend uses around 8GB of memory. We also released new versions of fs-sim and fs-api, and released strict-checked-vars.

UTxO-HD​

• Rebased UTxO-HD on top of node 8.2 (issue)
• Address new Maybe translations for UTxOs in Conway 1.6. (PR).
• Performed ad-hoc benchmarking syncs with both implementations. The LMDB backend uses around 8GB of memory.
• Fixed an infinite loop on the mempool tracing code that was preventing the node from shutting down gracefully.
• Fixed an infinite loop on the ledger state query for traversing queries that prevented the node from running QueryUTxOByAddress (PR).
• Measured the speed of QueryUTxOByAddress under different queryBatchSize values. There is a significant performance degradation for these queries when using the UTxO-HD version. Using the mainnet UTxO set at slot 90 million, querying an address took 40 seconds using the in-memory backend and 90 seconds using the LMDB backend, whereas the baseline took 7 seconds. We need to investigate if we can improve this situation or whether an external service that runs alongside the node is a better solution.
• Refactored and implemented ledgertable-related classes for the general HardforkBlock.

Genesis​

• Frisby and Esgen continue to engage with the Researchers on grinding against the Genesis design.

Fostering collaboration​

• Drafted a document explaining versioning of local state queries #273.

Support​

• Frisby is the release engineer this cycle.
• Successfully created work-in-progress ouroboros-consensus and cardano-api commits that integrate the 2023 Aug 7 tip of cardano-ledger. This will require a release of ouroboros-consensus, which hasn't happened yet.

High level summary​

The open fronts that the crypto team is working on are:

• Sidechains: Analysis of Halo2 SNARK verifier to plan a plutus implementation
• mithril: Full node verifier
• musig2: Finalised the missing fuzz tests
• kes_agent: engineer on PTO. Work halted

Low level summary​

Mithril​

• Work on a mithril flaky test #1105
• Keep working on Full Node Verifier #939. Should be merged in the coming week.

MuSig2​

• Finalised the fuzz tests #43

Sidechains​

• Analysis of Halo2 verifier. Necessary for a future implementation of a plutus verifier.

The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

The team is formalising mini protocols and also further developing the performance modelling prototype.

Details​

• Developing new framework for specification and verification of mini-protocols which is closer to the Haskell implementation.

• Developed a new internal representation for the DeltaQ algebra that allows for more modularity in backend implementations

• Discussions regarding the Cardano networking specification

High-level summary​

This week, the Hydra team achieved notable progress in various aspects of the project. The team updated the use case section for auctions on the /unstable branch of the website, improving the understanding of Hydra’s applicability.

From the development side, the team successfully completed event-sourced persistence, a key enhancement in the project’s architecture which improves off-chain transaction processing performance. They also added a “submit-transaction” endpoint to the API.

In addition to project-related progress, the team actively engaged in community reviews for several catalyst proposals related to Hydra and Mithril, contributing to the wider Cardano ecosystem.

Finally, the full report for the month of July was also published here.

What did the team achieve this week​

• Published the monthly report for July
• Updated the use case section for auctions (published on /unstable branch)
• Completed event sourced persistence #913
• Added a “submit-transaction” endpoint to the API #966
• Community reviews for several catalyst proposals related to Hydra and Mithril
• Created a network testing tool (hydra-net) #1006

What are the goals of next week​

• Update hydra-node to work with cardano-node version 8.x
• Remove the internal commit functionality
• Release version 0.12.0
• Update & streamline tutorial to work with latest version of hydra-node