Skip to main content

· 3 min read
John Lotoski

High level summary​

The SRE team continues work on cardano environment improvements and general environment maintenance.

Some notable recent changes, updates or improvements include:

  • Sanchonet environment was updated to 8.5.0-pre.
  • Cardano-parts now supports cardano-db-sync, cardano-smash, cardano-faucet and grafana cloud monitoring

Lower level summary​

Capkgs​

  • Various improvements and fixes including:
    • Add GHA cron schedule
    • Add nix auto-gc to avoid running out of storage during large package set builds
    • Add new packages to capkgs
    • Reduce runner storage requirement leaving more room for builds
    • Restructure capkgs attribute names to avoid package name collisions
    • Return to non-musl builds for cardano packages to retain journald compatability
    • Update cache usage to from file level to folder level to reduce network and latency overhead
    • Commit diff: capkgs-compare

Cardano-parts​

  • Updates cardano-node-ng to 8.5.0-pre and adds a Conway era automation job: cardano-parts-pull-16
  • Dbsync, smash, faucet and more: cardano-parts-pull-15
    • Adds cardano-db-sync, cardano-faucet, cardano-postgres, cardano-smash, profiles and/or services and related changes
    • Adds nginx vhost metrics exporter profile
    • Adds smash registered-relay-dump service and exporter for use until legacy relay nodes are scaled down
    • Adds bash *-ng autocompletion compatible wrappers
    • Adds a list-machines just recipe using nushell dataframe outer joins and scj ssh_config parser for fast cluster evals of machine state overview
    • Adds downstream grafana cloud dashboard as templates
    • Adds downstream grafana cloud alerts as templates
    • Updates grafana-agent profile with new exporter scrape hooks: cardano-db-sync, cardano-faucet, nginx-vts, varnish
    • Updates the basic profile with IOG cache and commonly used bins
    • Updates the pre-release profile to support cardano-db-sync, cardano-faucet, cardano-smash *-ng versioning
    • Updates flakeModule jobs with new conway era automation and additional IO encryption shimming and file type checks
    • Updates .sops.yaml template for supporting faucet secrets, workbench secrets, state-demo secrets
    • Updates the Justfile template with terraform fixes for workspace switching and provider auto-reconfiguration
    • Updates the cloudFormation terraformState template with stack modifications to preserve all resources in case of deletion
    • Updates the colmena template with dbsync, smash, faucet machines profiles and roles
    • Improves prior cardano-postgres modules to now automatically tune pg parameters based on machine cpuCount, memMiB and desired conns
    • Bumps capkgs node-ng to 8.5.0-pre

Cardano-playground​

  • Dbsync, smash, faucet and more: cardano-playground-pull-3
    • Adds a list-machines just recipe using nushell dataframe outer joins and scj ssh_config parser for fast cluster evals of machine state overview
    • Adds dbsync, smash, faucet machines and corresponding metrics exporters, dashboards and alerts
    • Moves the flake.cardano-parts.cluster.group attrSet name to groups to accurately reflect the plurality and the upstream corresponding change
    • Optimizes machine sizes
    • Updates .sops.yaml for supporting faucet secrets, workbench secrets, state-demo secrets
    • Updates the cloudFormation terraformState file with stack modifications to preserve all resources in case of deletion
    • Updates the cluster isNg definition to support cardano-db-sync, cardano-faucet, cardano-smash *-ng versioning
    • Updates the Justfile with terraform fixes for workspace switching and provider auto-reconfiguration

Cardano-world​

Sanchonet-demo​

  • Update for cardano-node 8.5.0, conway job recipes and cardano-parts interface changes: sanchonet-demo-commit

· One min read
Jean-Philippe Raynaud

High level overview​

This week, the Mithril team worked on the distribution of the Mithril client as a library that will be distributed on crates.io, the Rust dependencies repository. The team also worked on the deterministic computation of the transaction history of an address from the immutable files and started fixing a bottleneck in the aggregator that limits the ingestion of signatures.

Finally, they kept working on the activation of Cloudflare on the infrastructure and fixed some bugs in the Nix CI.

Low level overview​

  • Worked on the issue Make mithril-client crate a library #1141
  • Worked on the issue Compute deterministic Cardano transactions history PoC #1283
  • Worked on the issue ProtocolKeyRegistration creates performance bottleneck in aggregator #1187
  • Completed the issue Update Rust compiler in nix CI #1282
  • Worked on the issue Mithril networks infrastructure maintenance #1218
  • Worked on the issue Activate Cloudflare protection of infrastructure #1230

· 2 min read
Carlos LopezDeLara

2023-09-27 - 2023-10-10

High level summary​

CARDANO-NODE

CARDANO-CLI

  • Depracated governance query commands and moved all governance related queries to cardano-cli <era> query
  • Drep registration certificates now allow to include an anchor
  • Integrated Drep retirement ceriticates
  • clean up of Drep registration certificates.

CARDANO-API

  • Add support for committee hot key witnesses
  • Require conway onwards for voting
  • Conway drep registration: expose ledger anchor parameter
  • Add certs to txbody of Conway transactions

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

docs​

CI & project maintenance​

· One min read
Franco Testagrossa

High-level summary​

This week, the Hydra team released version 0.13.0, which includes fixes and extensions for state persistency and the ability to draft a commit transaction using inline datums.

The team also delivered the first version of the network resilience layer, significantly increasing head availability.

Additionally, they have been providing support to several funded Catalyst projects that want to build on Hydra.

Last but not least, they updated and published security policy and vulnerability disclosure policy reports on how to handle security vulnerabilities within Hydra.

What did the team achieve this week​

  • Release 0.13.0
  • Published vulnerability reports #1088
  • Merged network resilience work part I #1074
  • Planning for Cardano Summit participation
  • Discussions with funded Catalyst projects wanting to build on Hydra for support
  • Merged typos fix PR from @omahs #1095

What are the goals of next week​

  • Complete Aiken commit validator script #1072
  • Complete Kupo integration #1078
  • Brick upgrade on TUI #1103
  • Clean backlog
  • Prepare and rehearse demo and talk for Cardano Summit

· 3 min read
Michael Karg

High level summary​

  • Benchmarking: Continued benchmarking of UTxO-HD and performed benchmarks for the new tracing system.
  • Consensus QTAs: Our protoype approach is applied to potential regression fixes with GHC 9.2.7.
  • Development: We've developed strategies for future benchmarks of PlutusV3 and UTxO-HD's on-disk backing store.
  • Tracing: The machine-readable tracer configuration has been merged. Optimization of cardano-tracer started.
  • Nomad backend: Ongoing variance analysis and refined cluster topology.

Low level overview​

Benchmarking​

Performing and analyzing benchmarks for the UTxO-HD feature is an ongoing effort; we can reliably assess the performance of the in-memory backing store and evaluate possible optimizations (or regressions) for it.

Furthermore, benchmarks of our new tracing system after several rounds of optimization have been performed. The results show all key metrics now being unaffected by the choice of tracing system (legacy or new) - with the new system being able to provide more features and flexibility in comparison. The benchmarks also highlighted further points for optimization, with the focus now on the cardano-tracer service.

Consensus QTAs​

The Quantitative Timeliness Agreements (QTA) prototype is being used in coordination with Consensus and DevX to validate a series of patches that address optmization opportunities which GHC8.10 seizes, but GHC9.2 misses. The feedback from this approach is much more immediate than running benchmarks at system integration level. But once we eventually do, we expect to reproduce the relevant observations - which would mean a big step towards maturing the prototype.

Development​

Benchmarking UTxO-HD's on-disk backing store needs special attention: in virtualized environments, disk I/O is not a reliable metric as it passes several layers of indirection. As this is the very metric which will influence overall performance of this UTxO-HD flavour, we developed a plan to monitor such nodes, connected to a running network, on dedicated hardware - having direct SSD access. Replicating this setup for an entire benchmarking cluster of such nodes will be a future effort.

PlutusV3 will come with new builtins and a new cost model. It will take a specialized benchmark to ascertain the soundness of that model running a full cluster of nodes, possibly stressing expensive builtins. At the same time, we'd like to validate the many improvements that have gone into the Plutus evaluator.

Tracing​

The focus for further optimization of the new tracing system has shifted to cardano-tracer - the service receiving and processing traces from one (or more) nodes. Whilst undisputed that the code living in cardano-node is more performance critical, the receiving service must still minimize its resource footprint. Moreover, it can generate load for a running node when querying data points from it - which calls for tight control of that mechanism and its possible configurations.

Nomad backend​

Variance analysis of new nomad backend has revealed a necessary adjustment of the cluster's topology. We repeated the same analysis and now see even better confidence in the measurements taken with nomad. This concludes the work on the backend proper for the time being. The last steps before production use will focus on the interface between backend and our workbench, which provides all high-level benchmark definitions and analysis machinery.