Skip to main content

39 posts tagged with "sre"

View All Tags

· 3 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-world testnets of preprod, preview, sanchonet and some private test chains have largely completed their migration to the cardano-playground stack

Lower level summary​

Capkgs​

Cardano-parts​

  • General migration support PR for cardano-world to cardano-playground cardano-parts-pull-18
    • Iohk-nix and iohk-nix-ng were updated support the migration of cardano-world networks to cardano-playground
    • Sops-secrets dependent systemd services were fixed to ensure restart upon sops secrets changes
    • Db chain utilities (db-{analyser,synthesizer,truncater}) had -ng variants created to operate on both release and pre-release network chains
    • The profile-cardano-postgres nixos module received preset variables and prepared statements via pgsqlrc for faster and easier analysis of network state
    • The flakeModule jobs now has support for the cardano-cli era command in each of the job scripts by passing the $ERA_CMD variable
    • Default cardano-node-ng package is now 8.6.0-pre, dbsync on sanchonet is now sancho-2-2-0
    • For scripts using a nix-shell shebang, the cardano-parts devShell menu can be disabled from injecting itself into stdout by passing NOMENU=true
    • Template updates include:
      • Adds optional TF AZ declaration on ec2 resources
      • Adds a cardano node p2p dashboard to the grafana cloud stack
      • Adds a dbsync pool performance analysis query
      • Updates python distribute and delegation scripts from world for playground compatibility
      • Starts a python script lib to reduce shared code among the python scripts
      • Several justfile improvements and new recipes
    • More detail is available in the PR description
  • Update submit action script for 8.6 cardano-parts-pull-19
  • Update scripts for 8.6.0-pre cardano-parts-pull-21
    • Fixes subcommand names based on ERA_CMD
    • Adds deposits to some commands
    • Separates CC cold/hot key generation as host authorization has to occur after action is approved
    • CC voting enabled in vote job

Cardano-playground​

  • Migration PR to largely complete the network migration from cardano-world to cardano-playground: cardano-playground-pull-5
    • Adds re-spun private chain network
    • Migrates shelley-qa chain network from world
    • Justfile improvements and new recipes
    • Improve concurrent environment chain support
    • More detail is available in the PR description

Iohk-nix​

  • Migration to play: iohk-nix-pull-561
    • Migrate cardano-lib networks from world.dev.cardano.org to play.dev.cardano.org
    • Remove deprecated cardano-lib p2p network environment
    • Update sanchonet chain with respin changes
    • Update private chain with respin changes
    • Bump private and shelley-qa chains to sanchonet equivalent conway genesis
    • Bump preview, preprod chains to sanchonet equivalent conway genesis for node 8.6.0-pre pre-release testing

Sanchonet-demo​

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Cardano-world testnets of preprod, preview, sanchonet and some private test chains are in the process of being migrated to the cardano-playground stack

Lower level summary​

Capkgs​

  • Adds offchain-metadata-tools, dbsync sanchonet updates: capkgs-compare

Cardano-parts​

  • General package updates, modules improvements and template recipes to support network migration from world to playground cardano-parts-pull-17
    • Bumps cardano-db-sync-ng to sancho-2-0-0 tag
    • Bumps iohk-nix-ng to mig-sancho branch for sanchonet pool migration from world to play
    • Adds more machine system bins and devShell bins for scripting and debug purposes
    • Adds cardano-show-kes-period alias on any node machine importing profile-cardano-node-group module
    • Adds profile-cardano-node-topology module for a simplified interface to most common topology needs
    • Adds a job-delegate-rewards-stake-key job as an optional follow on to pool creation and registration jobs
    • Adds a topology function to filter self from group machines with an allowList for matching infixes
    • Adds metadata-server and related offchain-metadata-tools bins from capkgs
    • Updates justfile template with:
      • a new query-all recipe for getting status of multiple concurrent running environments
      • a new set-default-cardano-env recipe for fast switching between environments
      • a new start-demo recipe for forking a custom env into conway
      • a new start-node recipe for generic environment start
      • a new stop-node recipe for generic environment stop
      • updated list-machines recipe for handling of empty nixos machine config and empty ssh_config conditions
      • updated query-tip recipe to a generic query tip compatible with each environment

Cardano-playground​

· 3 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Sanchonet environment was updated to 8.5.0-pre.
  • Cardano-parts now supports cardano-db-sync, cardano-smash, cardano-faucet and grafana cloud monitoring

Lower level summary​

Capkgs​

  • Various improvements and fixes including:
    • Add GHA cron schedule
    • Add nix auto-gc to avoid running out of storage during large package set builds
    • Add new packages to capkgs
    • Reduce runner storage requirement leaving more room for builds
    • Restructure capkgs attribute names to avoid package name collisions
    • Return to non-musl builds for cardano packages to retain journald compatability
    • Update cache usage to from file level to folder level to reduce network and latency overhead
    • Commit diff: capkgs-compare

Cardano-parts​

  • Updates cardano-node-ng to 8.5.0-pre and adds a Conway era automation job: cardano-parts-pull-16
  • Dbsync, smash, faucet and more: cardano-parts-pull-15
    • Adds cardano-db-sync, cardano-faucet, cardano-postgres, cardano-smash, profiles and/or services and related changes
    • Adds nginx vhost metrics exporter profile
    • Adds smash registered-relay-dump service and exporter for use until legacy relay nodes are scaled down
    • Adds bash *-ng autocompletion compatible wrappers
    • Adds a list-machines just recipe using nushell dataframe outer joins and scj ssh_config parser for fast cluster evals of machine state overview
    • Adds downstream grafana cloud dashboard as templates
    • Adds downstream grafana cloud alerts as templates
    • Updates grafana-agent profile with new exporter scrape hooks: cardano-db-sync, cardano-faucet, nginx-vts, varnish
    • Updates the basic profile with IOG cache and commonly used bins
    • Updates the pre-release profile to support cardano-db-sync, cardano-faucet, cardano-smash *-ng versioning
    • Updates flakeModule jobs with new conway era automation and additional IO encryption shimming and file type checks
    • Updates .sops.yaml template for supporting faucet secrets, workbench secrets, state-demo secrets
    • Updates the Justfile template with terraform fixes for workspace switching and provider auto-reconfiguration
    • Updates the cloudFormation terraformState template with stack modifications to preserve all resources in case of deletion
    • Updates the colmena template with dbsync, smash, faucet machines profiles and roles
    • Improves prior cardano-postgres modules to now automatically tune pg parameters based on machine cpuCount, memMiB and desired conns
    • Bumps capkgs node-ng to 8.5.0-pre

Cardano-playground​

  • Dbsync, smash, faucet and more: cardano-playground-pull-3
    • Adds a list-machines just recipe using nushell dataframe outer joins and scj ssh_config parser for fast cluster evals of machine state overview
    • Adds dbsync, smash, faucet machines and corresponding metrics exporters, dashboards and alerts
    • Moves the flake.cardano-parts.cluster.group attrSet name to groups to accurately reflect the plurality and the upstream corresponding change
    • Optimizes machine sizes
    • Updates .sops.yaml for supporting faucet secrets, workbench secrets, state-demo secrets
    • Updates the cloudFormation terraformState file with stack modifications to preserve all resources in case of deletion
    • Updates the cluster isNg definition to support cardano-db-sync, cardano-faucet, cardano-smash *-ng versioning
    • Updates the Justfile with terraform fixes for workspace switching and provider auto-reconfiguration

Cardano-world​

Sanchonet-demo​

  • Update for cardano-node 8.5.0, conway job recipes and cardano-parts interface changes: sanchonet-demo-commit

· One min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Sanchonet environment was re-spun starting from slot 7171200 and updated to cardano-node 8.4.0-pre.
  • The use of cardano-node docker hub will be deprecated in preference of GHCR

Lower level summary​

Capkgs​

  • Refactor parsing scripts, add github action automation, various bugfixes and cleanup: capkgs-compare

Cardano-parts​

  • Updates secrets layout scheme, adds sops enc/dec for jobs, adds cloud monitoring profile, updates flake templates and other improvements/fixes: cardano-parts-pull-8

Cardano-playground​

  • Updates for new cardano-parts secrets handling and layout, TF workspace handling, group multivalue DNS support, grafana cloud monitoring and other improvements: cardano-playground

Cardano-world​

· 2 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent updates or improvements include:

  • Sanchonet environment was updated to 8.3.0-pre.
  • Cardano-parts now enables downstream consumers to spin up multiple cardano-networks per repository, an example of which is here.

Lower level summary​

Capkgs​

  • Add a readme, optimize flake inputs, improve nu and ruby parsing scripts: capkgs-compare

Cardano-ops​

Cardano-parts​

  • Utilize content addressed binaries to significantly improve eval, build and devShell performance: cardano-parts-pull-8
  • Add an aws flakeModule for obtaining ec2 specs, fix node.socket devShell error: cardano-parts-pull-10
  • Fix warns thrown during some builds and devShell usage: cardano-parts-pull-11
  • Add flakeModules and nixosModules in preparation for cardano-node grouped deployments: cardano-parts-pull-12
  • Enable group deployments by: adding several module, profile and role nixosModules; updating flakeModules; adding sops age secrets handling: cardano-parts-pull-13

Cardano-perf​

  • Modify the wireguard network from mesh to star, enable ICMP, enable Nomad raw_exec and add hydraJobs: cardano-perf-compare

Cardano-playground​

Cardano-world​