Skip to main content

26 posts tagged with "sre"

View All Tags

· 3 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node 8.9.2 is now deployed to mainnet, preprod, preview and shelley-qa environments.

  • Cardano-node 8.10.1-pre is now deployed to sanchonet and also to one-third of IOGs preprod environment nodes and two-thirds of IOGs preview environment nodes.

  • Private chain for Voltaire team was paused with plans for a future respin.

  • The network team's ouroboros-network-ops machine cluster was re-written using the cardano-parts stack to upgrade from the nixops/terraform/niv stack it was previously using.

Lower level summary​

Cardano-parts​

  • Sets cardano-node to 8.9.2, dbsync-ng to sancho-4.2.0; mithril to 2412.0, iohk-nix to include new peerSharing defaults and introduce a new block producer config. Adds a new truncate-chain recipe and improves mithril related services. More detail is available in the PR description: cardano-parts-pull-38

  • Sets cardano-node pre (-ng) to 8.10.1, dbsync to 13.2.0.2; mithril pre (-ng) to unstable, iohk-nix and iohk-nix-ng pin includes new Cardano Foundation bootstrap relays. Adds new aws machine management and other recipes, metadata job support for pool creation, misc fixes and improvements. More detail is available in the PR description: cardano-parts-pull-39

Cardano-mainnet​

  • Sets cardano-node to 8.9.2, mithril to 2412.0, iohk-nix to include new peerSharing defaults and colmena.nix cluster refactor for peerSharing adjustments and implements all updates in cardano-parts PR#38. See the PR description for more details: cardano-mainnet-pull-12

  • Sets dbsync to 13.2.0.2, iohk-nix and iohk-nix-ng to include new CF relays, adds new aws machine management recipes and implements all updates in cardano-parts PR#39. See the PR description for more details: cardano-mainnet-pull-13

Cardano-node​

  • Default peerSharing true and add block producer config to release binaries. See the PR description for more details: cardano-node-pull-5789

Cardano-ops​

Cardano-perf​

  • Adds a caddy webserver for run reviews and tunes the NVME FS mounts for performance: cardano-perf-compare

Cardano-playground​

  • Sets cardano-node to 8.9.2, mithril to 2412.0, iohk-nix to include new peerSharing defaults, refactors mdbook out of docs dir, adds example chain manipulation doc and implements all updates in cardano-parts PR#38. See the PR description for more details: cardano-playground-pull-21

  • Sets cardano-node pre (-ng) to 8.10.1, dbsync to 13.2.0.2, mithril pre (-ng) to unstable, iohk-nix and iohk-nix-ng to include new CF relays, adds a public TLS dbsync user connection option, stops private chain cluster and implements all updates in cardano-parts PR#39. See the PR description for more details: cardano-playground-pull-22

Iohk-nix​

Ouroboros-network-ops​

Sanchonet​

Sanchonet-demo​

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Private chain for Voltaire team was respun for cardano-node 8.10.0-pre pre-release and re-hard forked into Conway era.

  • Sanchonet was respun for cardano-node 8.10.0-pre pre-release and re-hard forked into Conway era.

  • Cardano-node 8.10.0-pre was also deployed to one-third of IOGs preprod environment nodes and two-thirds of IOGs preview environment nodes.

Lower level summary​

Cardano-parts​

  • Sets cardano-node|cli-ng to 8.10.0-pre, dbsync-ng to sancho-4.1.0; bumps nixpkgs (23.11) and nixpkgs-unstable; improves, adds new and fixes a number of just recipes; allows repo custom recipes to be kept as a separate import for easy justfile maintenance; refactors ip module checking into its own nixosModule and adds an extra abort option for safety. Much more detail is available in the PR description: cardano-parts-pull-36

Cardano-mainnet​

  • Bumps cardano-parts for a number of recipe additions, improvements, fixes. Fixes blockperf dnsmasq looksups and adjusts colmena topology code to account for intra-cluster localRoots now defaulting to trustable true and implements all updates in cardano-parts PR#36. See the PR description for more details: cardano-mainnet-pull-11

Cardano-perf​

  • Add a new perf-ssd machine class and deployment for ssd related performance testing and benchmarking. Add new just recipes and cluster resource tagging. cardano-perf-compare

Cardano-playground​

  • Bumps cardano-parts for a number of recipe additions, improvements, fixes, and node 8.10.0-pre. Respins private chain and sanchonet for node 8.10.0-pre with corresponding book updates and implements all updates in cardano-parts PR#36. See the PR description for more details: cardano-playground-pull-20

Iohk-nix​

  • WIP: Prepares block producer configurations to accomodate upcoming peerSharing default change: iohk-nix-pull-575

  • Provides iohk-nix updates for node 8.10: iohk-nix-pull-576

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node 8.9.1 is now deployed to all environments.

  • The legacy IOG mainnet metadata server has been retired with CF now providing metadata server services going forward.

  • Cardano-parts PR#35, merged and linked below, offers ip information integration into nixosConfiguration modules as well as template-diff and template-patch recipes for easier upgrades going forward.

Lower level summary​

Capkgs​

  • Adds a shortRev suffix to package names, fixes an rclone recipe, fixes a CI push action, defaults to recursively dereferenced object hashes, cleans up reference patterns. See the PR description for more details: capkgs-pull-2

Cardano-mainnet​

  • Bumps to cardano-node 8.9.1 and deploys all machines, makes ip information available in nixosCfgs, adds new expected machine alerts, tunes snapshot alerts and implements all updates in cardano-parts PR#35. See the PR description for more details: cardano-mainnet-pull-10

Cardano-ops​

Cardano-parts​

  • Upgrades cardano-node to 8.9.1 for both release and pre-release, integrates machine ip information into nixosConfigurations, enables /etc/hosts file usage in cardano-node topology, enhances cardano-node topology producer generation with customizable address types, introduces template patching recipes for easier cardano-parts updates to existing clusters. Much more detail is available in the PR description: cardano-parts-pull-35

Cardano-playground​

  • Bumps to cardano-node 8.9.1 and deploys all envs, rotates KES keys in most envs, makes ip information available in nixosCfgs and implements all updates in cardano-parts PR#35. See the PR description for more details: cardano-playground-pull-18

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node 8.9.0 is now deployed to all environments.

  • A new profile-cardano-db-sync-snapshots nixosModule is available for snapshotting dbsync state.

Lower level summary​

Cardano-mainnet​

  • Updates cardano node and cli pre-release (-ng) to 8.9.0. Adds a new cardano-db-sync snapshots server. Bumps mithril and enhances mithril usability with IOG trusted pool signature verification for clients and mithril verifier service for signers. Adds miscellaneous other small improvements, bug fixes and tuning. See the pull request description for more details: cardano-mainnet-pull-8

Cardano-node​

Cardano-ops​

  • Bumps to cardano-node 8.9.0, fixes metadata server module, removes the legacy snapshots server: cardano-ops-compare

Cardano-parts​

  • Updates cardano node and cli pre-release (-ng) to 8.9.0 and enables new bootstrapPeers features and compatibility in the topology functions. Bumps mithril and enhances mithril usability with IOG trusted pool signature verification for clients, mithril verifier service for signers, mithril capability for process-compose and node entrypoint jobs. Adds miscellaneous other small improvements, bug fixes and tuning. See the pull request description for more details: cardano-parts-pull-34

Cardano-playground​

  • Updates cardano node and cli pre-release (-ng) to 8.9.0. Bumps mithril and enhances mithril usability with IOG trusted pool signature verification for clients, mithril verifier service for signers, mithril capability for process-compose and node entrypoint jobs. Adds miscellaneous other small improvements, bug fixes and tuning. See the pull request description for more details: cardano-playground-pull-18

Govtool​

  • Fixes a coalesce function sql query bug in the govtool backend: govtool-commit

Iohk-nix​

  • Adds support for bootstrap peers and sanchonet mithril configuration: iohk-nix-pull-574

Sanchonet-demo​

  • Bumps cardano-parts for cardano-node 8.9.0 and brings the justfile demo recipes up to 8.9.0 compatibility: sanchonet-demo-compare

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node 8.8.1-pre is now deployed to sanchonet, two-thirds of preprod, one-third of preview and also voltaire private chain networks.

  • Catalyst Voltaire private chain was respun and an updated db-sync and govtool backend deployed after hard fork to Conway era.

  • IOG sanchonet pool block producers are now also signing mithril certificates.

  • A mithril-signer-verifier service now runs on all IOG SRE mithril signers and generates alerts if no recent mithril certificates have been signed.

  • All cardano-parts mithril clients, whether via nixosConfiguration, process-compose jobs or the nix cardano-node entrypoint, will now automatically verify that a trusted IOG pool has signed a mithril certificate prior to using the associated mithril state. If no trusted IOG pool has signed a recent mithril certificate, mithril state usage with be skipped and a classic sync from genesis to tip will be performed.

  • A significant amount of effort was applied towards the upcoming cardano-node 8.9.x-pre release in order to align nixos service module code between cardano-node nixos services, iohk-nix topology generation, cardano-ops legacy code, and cardano-parts module compatibility.

Lower level summary​

Work in progress​