High-level summary​

This week, the Hydra team worked on multiple fronts. They finished the investigation about the broken head on mainnet and re-opened their persistent head instance. The team also fixed the monthly report publication on their website and started sketching ideas and further improvements. Also, they are on the last mile to deliver a new feature which will allow parties to commit funds from extern wallets. Finally the team started to work on optimizing the performance on their benchmarks.

What did the team achieve this week​

• Finished investigation on broken head on mainnet #897 and re-opened it.
• Added support for externally committing regular utxo #887
• Fix monthly report publication on docs website and published the monthly report. Odd problems when publishing monthly report:
  • Make us think about if we should change something about the website #908
  • Open issue to docusaurus #9036
• Fixed a bug in the benchmark process #910
• Explored performance of the hydra-node{.verbatim} and identified a bottleneck.
• Timed transaction feature is being used by the auction project 🎉

What are the goals of next week​

• Complete performance analysis and start/plan improvements and provide regular benchmarks for Hydra #186
• Add hydra as tool to developer platform #872.
• Authenticate network messages #727.
• Complete journey for external commits using multiple script UTxOs #903
• Start implementing Option B for external commits #215.

High level summary​

We've made progress in all high level objectives

• CIP-1694 integration design
• UTxO-HD integration proof of concept
• schema migrations with a focus on speeding up frequent queries is part of release 13.1.1.2
• A new hire will help us with devx issues.

Lower level summary​

• We have an initial design for the Conway integration in db-sync
• The initial UTxO-HD integration resulted in a requested feuture that the consensus team is working on #88.
• Added two optional migrations that speed up querying UTxO #1416
• Work on improving the above feauture by adding tests and fixing tech debt #1429#1419
• Fixed issue with ada_pots being wrong https://github.com/input-output-hk/cardano-db-sync/pull/1407/commits/1ad4521a5601e8e98dc06bba2826d6b2f9b4fcf4
• Added a process which fixes in place scripts with wrong CBOR serialisation. https://github.com/input-output-hk/cardano-db-sync/pull/1407/commits/8792f72e2485b320de26c02dacc4d504d377ba17
• Prepared a tag 13.1.1.2 which is tested and ready to release. It contains many of the above.

The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

This sprint the team has been working on giving and preparing public talks, also on preparing final versions of their ICE 2023 papers, and on analysing chain selection.

This sprint the team has been working on final revisions for the pre-proceedings versions of two ICE 2023 papers.

Details​

• Lambda Days keynote by Kevin Hammond: https://www.lambdadays.org/lambdadays2023

• Codebeam keynote by Peter Van Roy: https://www.youtube.com/watch?v=h8sE3Ai8Dsk; https://www.youtube.com/watch?v=h8sE3Ai8Dsk

• Preparing talks and final versions for two ICE 2023 contributions: https://www.discotec.org/2023/ice

• Analysing chain selection.

• The team is hiring a performance engineering intern - https://apply.workable.com/io-global/j/BFEC5B0AFF/

High level summary​

The SRE team continues work on CI and cardano environment improvements. Some notable recent improvements include: expanding the darwin CI cluster and providing new aarch64 builder support; adding bare metal bitte cluster capability with network overlay for high IOPS workload performance, such as explorer.

Lower level summary​

Bitte​

• Equinix bare metal capability was added to bitte: bitte-pull-194
• Update bitte nixpkgs, nix version, nomad driver, equinix lifecycle, misc bug fixes: bitte-pull-201

Bitte-cells​

• Improvements made for patroni cluster recovery of large databases: bitte-cells-branch

Cardano-graphql​

• Update cardano-graphql and nixos service: cardano-graphql-pull-815

Cardano-node​

• Update cardano-node to use the updated iohk-nix environments: cardano-node-pull-5241

Cardano-ops​

• Update cardano-graphql, adds monitoring improvements and misc bug fixes: cardano-ops-pull-411
• Work in progress on mixed legacy and p2p topology clusters in cardano-ops: cardano-ops-branch

Cardano-world​

• Add a declarative cardano-faucet dashboard: cardano-world-commit
• Remove a requirement for root user in Nomad docker images: cardano-world-pull-68
• Enable explorer in cardano-world for high IOPS metal access: cardano-world-pull-69
• Migrate explorer cluster to cardano-world: cardano-world-pull-75
• Migrate testnet metadata server to cardano-world, cleanup envs: cardano-world-pull-77
• Add a faucet restore delegation script to cardano-world for non-functioning delegated pools: cardano-world-pull-79
• Integrate node 8.x into cardano-world: cardano-world-pull-84
• Update cardano-world to use iohk-nix for node environments: cardano-world-pull-87
• Create a stakepool analysis query for cluster pool performance overview: cardano-world-commit
• Added an iptables fixup script for broken bridged networking nomad jobs: cardano-world-commit
• Improve cardano-world job automation with additional parameterization: cardano-world-pr-92
• Update preview network experimental config defaults: cardano-world-pr-93

Ci-ops​

• Update legacy darwin builders and buildkite agent for ci-world network overlay and monitoring: ci-ops-pull-108

Ci-world​

• Resolve ci-world linux buildkite agent cleanup bug: ci-world-commit
• Integrate new darwin builders into ci-world: ci-world-pull-22
• Migrate patroni HA db to it's own namespace in ci-world: ci-world-pull-24

Cicero​

• Implement a cicero webhook backoff with exponential decay plus jitter: cicero-pull-79

Iohk-nix​

• Update iohk-nix to be source of truth for node environments: iohk-nix-pull-534
• Update preview network experimental config defaults: iohk-nix-pull-544

Openziti​

• Add darwin intel and arm support to ziti-edge-tunnel nix modules: openziti-bins-pull-1

High level summary​

Since the last update (2023-03-26) we did more improvements on our Test Framework, added more automated tests, kept analyzing and fixing nightly jobs failures, tested the 1.35.7 and 8.0.0 releases.

Workstreams​

Full list of closed PRs

Framework improvements:​

• Make cardano-submit-api REST API service available and run corresponding regression tests by default
• Update cabal build testing for 8.0.0
• Add support for testing governance SPO poll
• Use Poetry for Python dependency management
• Check status of known GH issues during tests runtime and finish the test accordingly
• Generate topology files with both IP addresses and DNS names
• Log issues like failure to start a cluster instance and report errors during tests runtime

Testing improvements​

• Add rollback testing:
  1. global consensus is reached after rollback, in situation where less than securityParam blocks were produced since cluster split
  2. global consensus is not reached when more than securityParam blocks were produced, and the result is permanent fork
• Add tests to governance SPO poll commands

Node:​

• tested the 1.35.7 release
• tested the 8.0.0 release

DB-Sync:​

• improvements in db-sync sync tests
• improvements in db-sync functional tests