Skip to main content

68 posts tagged with "sre"

View All Tags

· 3 min read
John Lotoski

High level summary​

The SRE team continues work on Cardano environment improvements and general maintenance.

Some notable recent changes, updates or improvements include:

  • New tracing system supporting PRs in iohk-nix and cardano-node have merged, making the new tracing system the default starting with upcoming node 10.6.0 release.

  • A dedicated aarch64-linux machine has been added to the devx-ci build cluster to improve CI performance for that arch.

  • Additional darwin resources will also be joining the devx-ci build cluster in the near future.

Repository Work -- Merged​

Cardano-mainnet​

  • Dbsync snapshots machine was switched from ramdisk based tmpfs to blockdevice tmpfs. Cardano-node pre-release has been updated to 10.5.0, cardano-cli pre-release to 10.11.0.0 and mithril to v2524.0. Opentofu resources, just recipes and other code has been updated to transition to use of ssh over AWS SSM with the closure of port 22 ingress. Includes various improvements with cardano-parts release v2025-06-24. Additional details are included in the PR header. cardano-mainnet-pr-36

Cardano-node​

  • This PR changes the default tracing system to the new tracing system and implements support for this where needed including: nixos cardano-node service, nixos cardano-tracer pre-existing service, nixos cardano-tracer new service, nixos scripts, nixos tests, docker, workbench service tracer changes, configuration reference files, bash scripts, CI. Additional details are included in the PR header. cardano-node-pr-6133

Cardano-parts​

  • Cardano-node pre-release has been updated to 10.5.0, cardano-cli pre-release to 10.11.0.0 and mithril to v2524.0. Opentofu resources, just recipes and other code has been updated to transition to use of ssh over AWS SSM with the closure of port 22 ingress. Additional details are included in the release notes. cardano-parts-release-v2025-06-24

Cardano-perf​

Cardano-playground​

  • Cardano-node pre-release has been updated to 10.5.0, cardano-cli pre-release to 10.11.0.0 and mithril to v2524.0. Opentofu resources, just recipes and other code has been updated to transition to use of ssh over AWS SSM with the closure of port 22 ingress. Includes various improvements with cardano-parts release v2025-06-24. Additional details are included in the PR header. cardano-playground-pr-48

Devx-ci​

  • Adds a new dedicated aarch64-linux hetzner rx170 machine to the CI cluster, including supporting nixos hardware and config module, re-keyed secrets, new wireguard ip range assignments based on machine namespace, bootstrap script updates and removal of emulated linux builds. Additional details are included in the PR header. devx-ci-pr-132

Iohk-nix​

  • This PR changes the default tracing system to the new tracing system and implements support for this where needed including: cardano-lib, generic log config, generic tracing config, testnet template config, legacy generic log config, legacy testnet template config. Additional details are included in the PR header. iohk-nix-pr-594

Repository Work In Progress -- PRs and Branches​

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on Cardano environment improvements and general maintenance.

Some notable recent changes, updates or improvements include:

  • Ouroboros-network-ops cluster was upgraded to the latest cardano-parts release with ssh over ssm usage.

  • The performance and tracing benchmarking cluster was upgraded to utilize ephemeral NVMe block devices for use in benchmarking ledger LMDB backend.

  • Blockperf, a software for measuring block propagation times in cardano networks, was updated to support the new tracing system.

  • Work to utilize the new tracing system as default for shipped configs such as in release artifacts, docker oci and cardano book as well as nixos module services continues and is near ready for merge. The cardano-node new tracing system related PRs currently listed in the "Repository Work In Progress" section are planned for merge prior to node 10.6.0 tagging.

Repository Work -- Merged​

Cardano-node​

  • Bumps iohkNix flake pin and updates cardano configurations to match for ci. cardano-node-pr-commit

  • Updates the nixos cardano-node service so that targetNumber* module option parameters inherit from ouroboros-network by default. cardano-node-pr-commit

Cardano-perf​

  • Upgrades the performances and tracing team's infra cluster to nixpkgs 25.05, latest colmena, fixes a wireguard startup race condition, and converts the cluster to usage of ephemeral NVMe volumes for LMDB capable benchmark testing on fast block devices. Details in the PR header. cardano-perf-pr-4

Iohk-nix​

  • Use ouroboros-network defaults for deadline and sync target related config. Update peer snapshots and useLedgerAfterSlot per env. iohk-nix-pr-601

Ouroboros-network-ops​

Repository Work In Progress -- PRs and Branches​

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on Cardano environment improvements and general maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano parts, cardano-playground and cardano-mainnet clusters were updated to nixpkgs 25.05 with nix 2.29.1 and all machines deployed.

  • Cardano-airgap repo now has the ISO available for download via hydra CI server.

  • The cardano-node new tracing system related PRs currently listed in the "Repository Work In Progress" section are planned for merge prior to 10.6.0 tagging. This will make the binary artifacts, docker images and book config default to use of the new tracing system.

Repository Work -- Merged​

Adawallet​

  • An adawallet maintenance PR, bumping flake inputs, repackaging cardano-hw-cli from source, fixing cardano-cli and cardano-hw-cli breaking changes in the new versions, and adding various features and improvements. Details in the PR header. adawallet-pr-21

Cardano-airgap​

  • An airgap maintenance PR, updating nixpkgs, integrating adawallet, fixing misc version breaking change issues, adding the ISO image to hydra and other various features and improvements. Details in the PR header. cardano-airgap-pr-4

Cardano-parts​

  • Sets nixpkgs to 25.05, and nix to 2.29.1. NixosModules and just recipes with breaking changes from those updates were fixed along with other miscellaneous improvements detailed below in the PR/release header. cardano-parts-release-v2025-06-05

Cardano-mainnet​

  • Nixpkgs has been updated to 25.05 and nix to 2.29.1. NixosModules and just recipes with breaking changes from those updates were fixed. Includes various improvements with cardano-parts release v2025-06-05. cardano-mainnet-pr-35

Cardano-node​

  • Bumps iohkNix flake pin to adjust QueryBatchSize declared default from 100 to 100000 to match the Consensus UTXO HD Migration Guide. Adjusts corresponding config files to match iohkNix update and for related CI jobs to pass. cardano-node-pr-6235

Cardano-playground​

  • Nixpkgs has been updated to 25.05 and nix to 2.29.1. NixosModules and just recipes with breaking changes from those updates were fixed. Includes various improvements with cardano-parts release v2025-06-05. cardano-playground-pr-47

Iohk-nix​

  • Adjust the QueryBatchSize declared default from 100 to 100000 to match the Consensus UTXO HD Migration Guide. iohk-nix-pr-600

Repository Work In Progress -- PRs and Branches​

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on Cardano environment improvements and general maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node and cardano-cli release versions are now set to 10.4.1 and 10.8.0.0, respectively, in cardano-parts release v2025-05-22.

  • The upcoming node 10.5 [pre-]release is being configured to use genesis mode by default on preview and preprod environments.

  • SRE is collaborating with IT Sec team to further harden our infra and clusters.

Repository Work -- Merged​

Cardano-parts​

  • Sets node release to 10.4.1, cardano-cli release to 10.8.0.0 and mithril to v2517.1. Adds mithril ancillary key support for signed volatile and ledger state. Switches to a blockPerf fork which can be used without publishing metrics to CF, ex: for testnet usage. Includes additional misc fixes and improvements described in the PR/release header. cardano-parts-release-v2025-05-22

Cardano-mainnet​

  • Sets node release to 10.4.1, cardano-cli release to 10.8.0.0 and mithril release v2517.1. Includes various improvements with cardano-parts release v2025-05-22. Deploys all machines to 10.4.1. cardano-mainnet-pr-34

Cardano-node​

  • Bumps iohkNix for 10.5 network param changes, preview/preprod default GenesisMode consensus config, per env useLedgerAfter and peer-snapshot updates. Updates CI for the iohkNix config changes. In the cardano-node nixos service, sets preview and preprod environments to use peer-snapshots. Enables genesis mode to work OOTB for preview/preprod on: docker images, binary artifacts, nixos built machines. cardano-node-pr-6225

Cardano-playground​

  • Sets node release to 10.4.1, cardano-cli release to 10.8.0.0 and mithril release v2517.1. Deploys blockPerf to all testnet machines without publishing metrics to CF. Includes various improvements with cardano-parts release v2025-05-22. Deploys all machines to 10.4.1. cardano-playground-pr-46

Iohk-nix​

  • Adds mithril network ancillary verification keys: the ancillary verification keys are a breaking change required to pull ancillary mithril state (ie: ledger and volatile state) in addition to immutable state. In earlier mithril releases, the ancillary state was bundled together with the immutable state. iohk-nix-pr-598

  • Node 10.5 updates: set GenesisMode for ConsensusMode as default on preview and preprod; tunes TargetNumberOfEstablishedPeers, and SyncTargetNumberOfEstablishedBigLedgerPeers per network team request; updates peer-snapshots per environment; updates useLedgerAfterSlot per environment. iohk-nix-pr-599

Repository Work In Progress -- PRs and Branches​

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on Cardano environment improvements and general maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node and cardano-cli pre-release versions are now set to 10.4.1 and 10.8.0.0, respectively, in cardano-parts release v2025-05-01.

  • In cardano-node, support for GHC 8.10 has been removed in master reducing CI eval times to < ~3 min. The next tagged pre-release or release after node 10.4.1 will support only GHC 9.6.

Repository Work -- Merged​

Cardano-parts​

Cardano-node​

  • Auto ghcr latest tagging currently occurs on full releases. Promotion of pre-release to release, however, doesn't trigger a published event, and instead requires a manual workflow dispatch off the newly promoted release tag. This PR adds ghcr latest tagging to the manual workflow dispatch pre-release to release use case. cardano-node-pr-6200

  • Drops support for haskellNix ghc 8.10.x which also reduces eval time to ~3 min with the removal of 249 variant packages. cardano-node-pr-6206

  • Removes the eventlogged package as ghc >= 9.4 includes unconditional -eventlog support in RTS. cardano-node-pr-6208

Cardano-playground​

  • Sets node pre-release to 10.4.1 and cardano-cli pre-release to 10.8.0.0 with cardano-parts release v2025-05-01. Updates and deploys cardano-book configuration for node pre-release 10.4.1. Adds preview1-test-a-1 machine, sops updates and secrets for test forging across legacy and new tracing. cardano-playground-pr-44

Repository Work In Progress -- PRs and Branches​