Skip to main content

SRE Team Update

· 3 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node 9.0.0 is now deployed to mainnet, preprod, preview, private and shelley-qa environments. The last several weeks have been very busy with pre-release and release activity and environment upgrades involving cardano-node versions 8.9.3, 8.9.4, 8.12.0-pre, 8.12.1, 8.12.2 and now 9.0.0 as of this update.

  • Sanchonet environment remains pinned at cardano-node version 8.11.0-pre until the next respin which will support 9.0.0 or greater.

  • Ogmios service and package options were added to cardano-parts.

  • Four documents were added to cardano-playground to better explain some operational procedures: debugging of peer-to-peer connections; governance voting with the playground stakepools; faucet setup; faucet pool de-delegation. Found at: docs/explain

  • One document was added to cardano-mainnet to explain cardano-snapshot operations. Found at: docs/explain

  • Private chain was stopped and re-spun with 2 hr epochs for testing.

  • Hydra and performance cluster machines had their configuration updated to be more robust to transient nix store caches outages which may re-occur in the future.

  • All machines in cardano-playground and cardano-mainnet clusters were updated to nixpkgs 24.05.

Lower level summary​

Cardano-mainnet​

  • Sets cardano-node to 8.12.2 as well as usage of a custom gc delay parameter branch for bootstrap nodes. Updates all machines to nixpkgs to 24.05 with openssh 9.8p1. Adds one new explainer readme document, new alerts and various script, recipe, and other improvements. See the PR description for more details: cardano-mainnet-pull-16

Cardano-ops​

  • Bumps to cardano-node 9.0.0, adds coredump metrics, adds OOM/coredump alerting, adjusts systemd stop timeout to avoid some unneccesary chain replays: cardano-ops-compare

Cardano-parts​

  • Sets cardano-node (release) and cardano-node-ng (pre-release) versions to 8.12.2 and cardano-db-sync-ng to sancho-5-0-0. Updates nixpkgs to 24.05. Includes nixosModule, dashboard, metric, alert and recipe improvements and new features. More detail is available in the PR description: cardano-parts-pull-43

Cardano-perf​

  • Adjusts nix config to avoid R2 500 errors on transient cache problems and adds explorer to perf class: cardano-perf-compare

Cardano-playground​

  • Sets cardano-node (release) and cardano-node-ng (pre-release) versions to 8.12.2 and cardano-db-sync-ng to sancho-5-0-0. Updates all machines to nixpkgs to 24.05 with openssh 9.8p1. Respins private chain and KES rotates multiple chains. Adds four new explainer readme documents, new alerts and various script, recipe, and other improvements. See the PR description for more details: cardano-playground-pull-27

Iohk-nix​

Ops-lib​

  • Updates deployers with recent nixpkgs, nix, refactors to preserve legacy nixops usage, adds starship and fzf: ops-lib-pull-134

  • Bumps openssh to 9.8p1 ops-lib-pull-135