High level overview​

This week, the Mithril team released the new distribution 2430.0 which includes support for Cardano node version 9.1.0. They activated the certification of Cardano transactions in the release-preprod and pre-release-preview networks, and also included bug fixes and performance improvements. The team continued their work on the certification of Cardano stake distribution, activated the certification in the test networks, and started working on implementing new routes for the aggregator REST API..

Finally, the team benchmarked the signer's footprint on the SPO infrastructure, thoroughly tested edge cases for Cardano transaction certification, and continued drafting a CIP for the diffusion of Mithril signatures through the Cardano network.

Low level overview​

• Released the new distribution 2430.0
• Publication of a dev blog post about the progressive activation of the certification of Cardano transactions
• Completed the issue Release 2430 distribution #1830
• Completed the issue Implement signable and artifacts builders for Cardano Stake Distribution #1832
• Completed the issue Refactor SlotNumber type #1851
• Completed the issue Slow import of Cardano transactions in testing-mainnet #1849
• Completed the issue ChainReader is stuck in Agency is theirs error #1836
• Completed the issue Make SignedEntityType in RegisterSignatureMessage non optional #1863
• Completed the issue Upgrade Cardano node 9.1.0 #1855
• Completed the issue Typos in repository #1862
• Worked on the issue Benchmark signer footprint on SPO infrastructure #1826
• Worked on the issue CIP for Mithril signature diffusion through Cardano network #1775
• Worked on the issue Test Cardano transaction chain rollbacks #1840
• Worked on the issue Implement routes for Cardano Stake Distribution in aggregator REST API #1841
• Worked on the issue Duplicate page on /index.html of website #1861

High level summary​

The SRE team continues work on cardano environment improvements and general maintenance.

Some notable recent changes, updates or improvements include:

• Our new baseline version of Cardano Node is 9.1.0 and all environments have been updated. The main change from node 9.0.0 is that node 9.1.0 requires a Conway genesis file at startup, where the genesis file was optional in node 9.0.0.

• Sanchonet had another respin for node 9.0.0 with new Conway genesis parameters and has since been upgraded to node 9.1.0.

• The cardano-monitoring cluster received a lot of documentation and improvements and now also serves as the home for devx-ci metrics.

• Hydra CI performance was improved once again by changes to our custom Nix evaluator. We also found further ways to improve the resource usage of waiting for IFDs.

Cardano Airgap​

A new project that provides a completely airgapped environment for constitution members to sign proposals and transactions. It consists of an image for USB sticks and ensures all private data is stored securely with strong encryption.

We'll keep it updated as required with the latest Cardano versions.

cardano-airgap

Cardano Parts​

• cardano-node and cardano-node-ng are now at version 9.0.0
• cardano-db-sync and cardano-db-sync-ng are now at version 13.3.0.0
• Several NixOS modules and recipes have been fixed and improved.
• Bump dependency of capkgs for node, db-sync, mithril, and cardano-wallet updates.
• Update profile-cardano-db-sync-snapshots for schema 13.3 docs and with script edge case fixes
• Update profile-cardano-node-group to use a SIGINT instead of SIGTERM for systemd stop
• Update profile-common to deploy atd service
• Update template recipe dbsync-prep to match faucet script defaults
• Update template recipe update-ips to fix a nushell breaking change
• Update .envrc with a newer direnv version and allows for symlinks on .envrc.local and ~/.age/credentials

PR#44

Cardano Playground​

• All networks are now running cardano-node 9.1.0 in preparation of the Chang hard-fork.
• Also upgraded db-sync to 13.3.0.0
• Added the cardano-ipfs module and a derivation for pinata-go-cli that is used to store and distribute documents that can be referenced on chain.
• Some updates to the Cardano Operations Book about:
  • UseLedgerPeerAfter updates
  • Sanchonet respins configs
  • Dbsync EnableFutureGenesis flag
• Add a block header block producer readiness test
• Respin of sanchonet for node 9.0.0, then upgraded to 9.1.0
• Tune webserver size and Varnish RAM to improve caching efficiency
• Updates govtool module for multi-nginx module compatibility
• Updates update-ips recipe for nushell breaking change in nixpkgs 24.05
• Updates direnv version, allow symlinks config files used by direnv
• Update .envrc with a newer direnv version, allow symlinks on direnv used config files

PR#28

Cardano Mainnet​

• Upgraded Cardano Node to 9.0.0
• Upgraded Cardano DB Sync to 13.3.0.0
• Bump capkgs dependency
• Investigate bootstrap missingBlock error and deploy fixes for it.
• Update scripts to be compatible with latest nushell version

PR#17

Cardano Monitoring​

• Write comprehensive documentation for all the Nix code, as well as detailed instructions for usage and deployment.
• Overhaul most Just tasks to bring them more in sync with the other repositories
• Upgrade all machines to NixOS 24.05
• Upgrade auth-keys-hub to prevent lockout in case SOPS is unable to decrypt
• Fix SOPS decryption failure on boot because of missing network.
• Limit bootloader entries to 5 since the /boot partition is tiny
• Additionally add fallback SSH keys for emergency use

PR#1

IOHK Nix​

• Update ledger peers to be after a more recent epoch boundary to improve bootstrapping and fix a private chain p2p delayed sync config issue.
• Update sanchonet conway-genesis for respin
  • DRep voting thresholds both need to be 65%
  • Set govActionLifetime above the guardrail because of the short epochs.
  • Set minCommitteeSize to 5 (from recommended 7) because only 5 ICC members were able to provide keys for the respin.

PR#584 PR#585

Cardano Ops​

• tweak the stop timeout and change Cardano Node killsignal to SIGINT for clean restarts.

Diff

CAPkgs​

Added following packages:

• cardano-node 9.0.0 and 9.1.0
• For cardano-db-sync releases of sancho-5.1.0 and 13.3.0.0
• For mithril 2428.0 and mithril-unstable
• For cardano-wallet v2024-07-19

High-level summary​

This week we merged incremental decommits, and got the latest Hydra Head running, on preview, between the team. We also provided a new NixOs configuration for running a Hydra node. We will be working on incremental commits, followsup to the incremental decommit work, and improvements/tests for our network reliability next.

What did the team achieve?​

• Incremental decommit merged! #1344
• Incremental decommit in the TUI #1518
• Hydra Head, and Hydraw, up and running on preview network
• New NixOS-based setup for a Hydra Head node #1506

What's next?​

• Continued work on incremental commit: Off-chain journey
• Network reliability investigations
• Small followups to incremental decommit work
• Support Hydra demo at Rare Evo
• Have a Head open on preview to see it working through the upcoming hard fork

High level overview​

This week, the Mithril team focused on preparing for the upcoming release of Cardano transaction certification in Mithril networks. They fixed a bug that was preventing the import of transactions close to the tip of the chain, investigated performance issues on imports from mainnet and SanchoNet, refactored some types used for block and slot numbers, and thoroughly benchmarked performance and tested edge cases.

Finally, the team continued working on the certification of the Cardano stake distribution and redacting a CIP for the diffusion of Mithril signatures through the Cardano network.

Low level overview​

• Completed the issue Cleanup Immutable File in Cardano transaction #1825
• Completed the issue Upgrade testing-sanchonet for respin with Cardano 9.0.0 #1822
• Completed the issue testing-sanchonet does not sign Cardano transactions #1845
• Completed the issue Refactor BlockNumber type #1755
• Completed the issue Cleanup Immutable File in Cardano transaction #1825
• Completed the issue Make Certificate messages beacon field optional #1807
• Worked on the issue Implement signable and artifacts builders for Cardano Stake Distribution #1832
• Worked on the issue CIP for Mithril signature diffusion through Cardano network #1775
• Worked on the issue Test Cardano transaction chain rollbacks #1840
• Worked on the issue Refactor SlotNumber type #1851
• Worked on the issue Slow import of Cardano transactions in testing-mainnet #1849
• Worked on the issue ChainReader is stuck in Agency is theirs error #1836
• Worked on the issue Benchmark signer footprint on SPO infrastructure #1826

High level summary​

• Benchmarking: Release benchmarks for Node 9.0; Plutus execution budget scaling benchmarks.
• Development: Improved shutdown behaviour for tx-generator merged to master. Work on governance action benchmarking workload is ongoing.
• Workbench: Haskell profile content definition merged to master.
• Tracing: Factoring out RTView was merged to master. Work on metrics naming ongoing, minimizing migration effort.
• Consensus QTAs: Design for automating and data warehousing beacon measurements is complete.

Low level overview​

Benchmarking​

Runs and analyses for a full set of release benchmarks have been performed - and published - for Node version 9.0.0. Comparing with the latest mainnet release 8.12.1, we could not observe any performance regression. 9.0.0 exhibits an improvement in Block Fetch duration, which results in slightly better overall network performance.

Additionally, we've performed scaling benchmarks of Plutus execution budgets. In these series of benchmarks, we measure the performance impact of changes to those budgets in the protocol parameters. Steps (CPU) and memory budgets are scaled independently of each other, and performance testing takes place using Plutus scripts that either are heavy on allocations but light on CPU, or vice versa. These performance tests are meant to explore the headroom of those budgets, taking into account cost model changes, and recent optimization capabilites of the Plutus compiler.

Development​

Our workload submission service tx-generator has been equipped with the ability to handle POSIX signals for graceful shutdown scenarios. Furthermore, as it is highly concurrent, error reporting on a per-thread basis has been added, enhancing feedback from the service. Along with some quality-of-life improvements, these changes have landed in master.

The Conway governance action workloads for benchmarking have completed design phase, and we've settled on an implementation plan. Implementation work itself has started.

Workbench​

Generating the contents for any benchmarking profile has now become a dedicated Haskell tool, cardano-profile, which has landed in master. Adding type safety and a test suite to profile definitions is a major improvement over shell code that directly manipulates JSON objects. Furthermore, it makes reliably modifying, or creating, benchmarking profiles much more accessible to engineers outside of our team.

Tracing​

With factoring out RTView, and making it an opt-in component of the cardano-tracer build, we've reduced the service's dependency graph significantly. Furthermore, the service has become more lightweight on resources. We'll continue to maintain RTView, and guarantee it will remain buildable and usable in the future.

Aligning metrics naming and semantics of new and legacy tracing is ongoing work. This task is part of a larger endeavour to minimize the effort necessary for users to migrate to the new system.

Consensus QTAs​

beacon, which currently measures performance of certain ledger operations on actual workload fragments, is a first step in building a benchmarking framework based on Delta-Q system design and quantitative timeliness agreements. We've finished the design of how to automate those measurements at sensible points in time, and provide a storage schema which will enable access and analysis that fits with the framwork.