High-level summary​

This week, the Hydra team updated hydra-node to support cardano-node version 8.1.2, ensuring compatibility with the latest mainnet release. The team also participated in discussions about a "cardanonical" JSON schema and added it as a submodule to the project, contributing to improved data modeling and interoperability.

In terms of community engagement, the team successfully onboarded a new contributor, also participated in a RareEvo Twitter space, and continued preparations for the Hydra master-class.

What did the team achieve this week​

• Updated dependencies to support cardano-node 8.1.2 #1007
• Engaged in discussions about a “cardanonical” json schema and add it as submodule to the project #1013
• Onboarded a new contributor to the hydra project (@locallycompact)
• Joined RareEvo twitter space and continued preparation of hydra master-class

What are the goals of next week​

• Update & streamline tutorial to work with latest version of hydra-node
• Remove the internal commit functionality #954
• Release 0.12.0

High level summary​

• Benchmarking: We've concluded benchmarking node version 8.2.0.
• Tracing: Optimization of the new tracing system has been merged; we're currently working on self-documenting tracing configuration.
• Nomad backend: A PR that makes our backend take advantage of added flexibility of the new hardware cluster has been merged.

Low level overview​

Benchmarking​

As part of our release benchmarking cycle, we've completed and analysed the runs for the 8.2.0 version of cardano-node. In addition to the adjustment of sanity checks in our automation, we had to implement small changes in the analysis pipeline as well to accomodate the new version.

Tracing​

A significant amount of optimizations for the new tracing system has finally been merged to master. At the moment, we're working on having a trace message self-document the final tracing configuration of a running node. Apart from adding insight into the system, this feature also aims at making future hot reloading of tracing configuration explicit and straightforward.

Furthermore, we're setting up a final round of system integration level benchmarks comparing new against legacy tracing.

Nomad backend​

The new hardware cluster permits greater flexibility as far as SSH access is concerned. By using nomad for a consistent and reliable deployment, but taking advantage of direct connections for healthchecks and data transfer we believe we were able to reduce overall network latency in the nomad cluster. This improves confidence when capturing all network related measurements during our benchmarks.

A PR that adds these capabalities to our nomad backend - along with very many quality-of-life improvements - has been merged to master.

High level overview​

The Mithril team completed the Mithril protocol’s mainnet beta launch: the release-mainnet network is now consistently producing new certificates and snapshots! The team has released a new distribution 2331.1 which includes some optimizations and bug fixes: the bootstrap time of a Cardano node restored from a Mithril snapshot has been significantly reduced, and a bug that made the aggregator produce corrupted archives has been fixed. They also kept working on the implementation of the stress test tool for benchmarking the aggregator: the first phase has been implemented, as well as a monitoring feature. Additionally, they kept implementing the refactoring of the serialization/deserialization of the entities of the cryptographic library, and working on the post deployment of the mainnet infrastructure.

Finally, they completed the rewriting and enhancements of the documentation website, and they completed the runbook for operating a production Mithril network.

Low level overview​

• Released the new distribution 2331.1
• Completed the epic Release 'mainnet' Mithril network #918
• Worked on the epic Benchmark performances of Mithril Aggregator #904:
  • Completed the issue Design & implement basic stress test tool for aggregator - phase 1 #991
• Worked on the epic Post deployment 'mainnet' infrastructure #1091:
  • Completed the issue Create Production Infrastructure Runbook #1085
  • Completed the issue Ingest 'mainnet' metrics/logs in Grafana #1122
• Worked on optimizations:
  • Completed the issue Add a 'clean' file to extracted database in client #1131
  • Worked on the issue Create explorer page with registered signers #1097
• Worked on refactoring:
  • Worked on the issue Refactor (de)serialization of crypto entities #668
• Worked on bugs:
  • Completed the issue Client fails to unpack some snapshot archive #1137
  • Completed the issue The 'release-preprod' network stopped producing certificates #1114
  • Completed the issue The 'release-mainnet' network does not show up in the explorer #1111
• Worked on dependencies:
  • Completed the issue Upgrade Cardano node to '8.1.2' #1090
• Worked on troubleshooting:
  • Completed the issue Signer can't sign with 'Invalid signature for party' error #1125

High level summary​

This week the Consensus team worked on integrating the latest node release into the UTxO HD branch. Our ad-hoc benchmarks in which we sync a node from scratch showed that the LMDB backend uses around 8GB of memory. We also released new versions of fs-sim and fs-api, and released strict-checked-vars.

UTxO-HD​

• Rebased UTxO-HD on top of node 8.2 (issue)
• Address new Maybe translations for UTxOs in Conway 1.6. (PR).
• Performed ad-hoc benchmarking syncs with both implementations. The LMDB backend uses around 8GB of memory.
• Fixed an infinite loop on the mempool tracing code that was preventing the node from shutting down gracefully.
• Fixed an infinite loop on the ledger state query for traversing queries that prevented the node from running QueryUTxOByAddress (PR).
• Measured the speed of QueryUTxOByAddress under different queryBatchSize values. There is a significant performance degradation for these queries when using the UTxO-HD version. Using the mainnet UTxO set at slot 90 million, querying an address took 40 seconds using the in-memory backend and 90 seconds using the LMDB backend, whereas the baseline took 7 seconds. We need to investigate if we can improve this situation or whether an external service that runs alongside the node is a better solution.
• Refactored and implemented ledgertable-related classes for the general HardforkBlock.

Genesis​

• Frisby and Esgen continue to engage with the Researchers on grinding against the Genesis design.

Fostering collaboration​

• Drafted a document explaining versioning of local state queries #273.

Support​

• Frisby is the release engineer this cycle.
• Successfully created work-in-progress ouroboros-consensus and cardano-api commits that integrate the 2023 Aug 7 tip of cardano-ledger. This will require a release of ouroboros-consensus, which hasn't happened yet.

High level summary​

The open fronts that the crypto team is working on are:

• Sidechains: Analysis of Halo2 SNARK verifier to plan a plutus implementation
• mithril: Full node verifier
• musig2: Finalised the missing fuzz tests
• kes_agent: engineer on PTO. Work halted

Low level summary​

Mithril​

• Work on a mithril flaky test #1105
• Keep working on Full Node Verifier #939. Should be merged in the coming week.

MuSig2​

• Finalised the fuzz tests #43

Sidechains​

• Analysis of Halo2 verifier. Necessary for a future implementation of a plutus verifier.