Skip to main content

42 posts tagged with "sre"

View All Tags

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • A preview of the new cardano tracing system is available by using the profile-cardano-node-new-tracing new nixosModule from cardano-parts and is complemented with new tracing system dashboards and alerts

  • Custom cardano metrics can be pushed to a statsd server and then scraped to a monitoring solution using the profile-cardano-custom-metrics new nixosModule from cardano-parts

  • A new cardano-parts job for spinning up clusters using a recent cardano-cli command, create-testnet-data, was implemented: .#job-gen-custom-node-config-data

Lower level summary​

Cardano-faucet​

Cardano-parts​

  • Sets cardano-faucet-ng to 8.10, and mithril[-ng] to 2418-1. A preview nixosModule for cardano-node's new tracing system has been added as well as a cardano custom-metrics module. Dashboards have been added for the new tracing system. Cardano-node alerts have been refactored into several rulesets which include some new alerts as well as new tracing system alerts. A job for spinning a cardano cluster up using a new underlying cardano-cli mechanism, create-testnet-data, was added. More detail is available in the PR description: cardano-parts-pull-40

Cardano-mainnet​

  • Sets mithril to 2418-1. Cardano-node alerts have been refactored into several rulesets which include some new alerts as well as new tracing system alerts. See the PR description for more details: cardano-mainnet-pull-14

Cardano-perf​

Cardano-playground​

  • Sets cardano-faucet-ng to 8.10 which fixes a sanchonet faucet instability, and mithril[-ng] to 2418-1. Adds a new cardano-node tracing system preview to some machines with corresponding dashboards. Cardano-node alerts have been refactored into several rulesets which include some new alerts as well as new tracing system alerts. A job for spinning a cardano cluster up using a new underlying cardano-cli mechanism, create-testnet-data, was added and integrated with the Justfile start-demo recipe. Includes other misc tuning and improvements. See the PR description for more details: cardano-playground-pull-23

· 3 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node 8.9.2 is now deployed to mainnet, preprod, preview and shelley-qa environments.

  • Cardano-node 8.10.1-pre is now deployed to sanchonet and also to one-third of IOGs preprod environment nodes and two-thirds of IOGs preview environment nodes.

  • Private chain for Voltaire team was paused with plans for a future respin.

  • The network team's ouroboros-network-ops machine cluster was re-written using the cardano-parts stack to upgrade from the nixops/terraform/niv stack it was previously using.

Lower level summary​

Cardano-parts​

  • Sets cardano-node to 8.9.2, dbsync-ng to sancho-4.2.0; mithril to 2412.0, iohk-nix to include new peerSharing defaults and introduce a new block producer config. Adds a new truncate-chain recipe and improves mithril related services. More detail is available in the PR description: cardano-parts-pull-38

  • Sets cardano-node pre (-ng) to 8.10.1, dbsync to 13.2.0.2; mithril pre (-ng) to unstable, iohk-nix and iohk-nix-ng pin includes new Cardano Foundation bootstrap relays. Adds new aws machine management and other recipes, metadata job support for pool creation, misc fixes and improvements. More detail is available in the PR description: cardano-parts-pull-39

Cardano-mainnet​

  • Sets cardano-node to 8.9.2, mithril to 2412.0, iohk-nix to include new peerSharing defaults and colmena.nix cluster refactor for peerSharing adjustments and implements all updates in cardano-parts PR#38. See the PR description for more details: cardano-mainnet-pull-12

  • Sets dbsync to 13.2.0.2, iohk-nix and iohk-nix-ng to include new CF relays, adds new aws machine management recipes and implements all updates in cardano-parts PR#39. See the PR description for more details: cardano-mainnet-pull-13

Cardano-node​

  • Default peerSharing true and add block producer config to release binaries. See the PR description for more details: cardano-node-pull-5789

Cardano-ops​

Cardano-perf​

  • Adds a caddy webserver for run reviews and tunes the NVME FS mounts for performance: cardano-perf-compare

Cardano-playground​

  • Sets cardano-node to 8.9.2, mithril to 2412.0, iohk-nix to include new peerSharing defaults, refactors mdbook out of docs dir, adds example chain manipulation doc and implements all updates in cardano-parts PR#38. See the PR description for more details: cardano-playground-pull-21

  • Sets cardano-node pre (-ng) to 8.10.1, dbsync to 13.2.0.2, mithril pre (-ng) to unstable, iohk-nix and iohk-nix-ng to include new CF relays, adds a public TLS dbsync user connection option, stops private chain cluster and implements all updates in cardano-parts PR#39. See the PR description for more details: cardano-playground-pull-22

Iohk-nix​

Ouroboros-network-ops​

Sanchonet​

Sanchonet-demo​

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Private chain for Voltaire team was respun for cardano-node 8.10.0-pre pre-release and re-hard forked into Conway era.

  • Sanchonet was respun for cardano-node 8.10.0-pre pre-release and re-hard forked into Conway era.

  • Cardano-node 8.10.0-pre was also deployed to one-third of IOGs preprod environment nodes and two-thirds of IOGs preview environment nodes.

Lower level summary​

Cardano-parts​

  • Sets cardano-node|cli-ng to 8.10.0-pre, dbsync-ng to sancho-4.1.0; bumps nixpkgs (23.11) and nixpkgs-unstable; improves, adds new and fixes a number of just recipes; allows repo custom recipes to be kept as a separate import for easy justfile maintenance; refactors ip module checking into its own nixosModule and adds an extra abort option for safety. Much more detail is available in the PR description: cardano-parts-pull-36

Cardano-mainnet​

  • Bumps cardano-parts for a number of recipe additions, improvements, fixes. Fixes blockperf dnsmasq looksups and adjusts colmena topology code to account for intra-cluster localRoots now defaulting to trustable true and implements all updates in cardano-parts PR#36. See the PR description for more details: cardano-mainnet-pull-11

Cardano-perf​

  • Add a new perf-ssd machine class and deployment for ssd related performance testing and benchmarking. Add new just recipes and cluster resource tagging. cardano-perf-compare

Cardano-playground​

  • Bumps cardano-parts for a number of recipe additions, improvements, fixes, and node 8.10.0-pre. Respins private chain and sanchonet for node 8.10.0-pre with corresponding book updates and implements all updates in cardano-parts PR#36. See the PR description for more details: cardano-playground-pull-20

Iohk-nix​

  • WIP: Prepares block producer configurations to accomodate upcoming peerSharing default change: iohk-nix-pull-575

  • Provides iohk-nix updates for node 8.10: iohk-nix-pull-576

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node 8.9.1 is now deployed to all environments.

  • The legacy IOG mainnet metadata server has been retired with CF now providing metadata server services going forward.

  • Cardano-parts PR#35, merged and linked below, offers ip information integration into nixosConfiguration modules as well as template-diff and template-patch recipes for easier upgrades going forward.

Lower level summary​

Capkgs​

  • Adds a shortRev suffix to package names, fixes an rclone recipe, fixes a CI push action, defaults to recursively dereferenced object hashes, cleans up reference patterns. See the PR description for more details: capkgs-pull-2

Cardano-mainnet​

  • Bumps to cardano-node 8.9.1 and deploys all machines, makes ip information available in nixosCfgs, adds new expected machine alerts, tunes snapshot alerts and implements all updates in cardano-parts PR#35. See the PR description for more details: cardano-mainnet-pull-10

Cardano-ops​

Cardano-parts​

  • Upgrades cardano-node to 8.9.1 for both release and pre-release, integrates machine ip information into nixosConfigurations, enables /etc/hosts file usage in cardano-node topology, enhances cardano-node topology producer generation with customizable address types, introduces template patching recipes for easier cardano-parts updates to existing clusters. Much more detail is available in the PR description: cardano-parts-pull-35

Cardano-playground​

  • Bumps to cardano-node 8.9.1 and deploys all envs, rotates KES keys in most envs, makes ip information available in nixosCfgs and implements all updates in cardano-parts PR#35. See the PR description for more details: cardano-playground-pull-18

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-node 8.9.0 is now deployed to all environments.

  • A new profile-cardano-db-sync-snapshots nixosModule is available for snapshotting dbsync state.

Lower level summary​

Cardano-mainnet​

  • Updates cardano node and cli pre-release (-ng) to 8.9.0. Adds a new cardano-db-sync snapshots server. Bumps mithril and enhances mithril usability with IOG trusted pool signature verification for clients and mithril verifier service for signers. Adds miscellaneous other small improvements, bug fixes and tuning. See the pull request description for more details: cardano-mainnet-pull-8

Cardano-node​

Cardano-ops​

  • Bumps to cardano-node 8.9.0, fixes metadata server module, removes the legacy snapshots server: cardano-ops-compare

Cardano-parts​

  • Updates cardano node and cli pre-release (-ng) to 8.9.0 and enables new bootstrapPeers features and compatibility in the topology functions. Bumps mithril and enhances mithril usability with IOG trusted pool signature verification for clients, mithril verifier service for signers, mithril capability for process-compose and node entrypoint jobs. Adds miscellaneous other small improvements, bug fixes and tuning. See the pull request description for more details: cardano-parts-pull-34

Cardano-playground​

  • Updates cardano node and cli pre-release (-ng) to 8.9.0. Bumps mithril and enhances mithril usability with IOG trusted pool signature verification for clients, mithril verifier service for signers, mithril capability for process-compose and node entrypoint jobs. Adds miscellaneous other small improvements, bug fixes and tuning. See the pull request description for more details: cardano-playground-pull-18

Govtool​

  • Fixes a coalesce function sql query bug in the govtool backend: govtool-commit

Iohk-nix​

  • Adds support for bootstrap peers and sanchonet mithril configuration: iohk-nix-pull-574

Sanchonet-demo​

  • Bumps cardano-parts for cardano-node 8.9.0 and brings the justfile demo recipes up to 8.9.0 compatibility: sanchonet-demo-compare