High level summary​

The Developer Experience team has been devoted to day-to-day troubleshooting and support of various elements including build failures, compiler upgrades, the maintaince of our cardano-haskell-packages (CHaP), and infrastructure like GitHub Actions, iohk-nix, haskell.nix, and devx. Furthermore, we have also contributed to upstream tooling improvements.

Lower level summary​

build support & maintainance​

Our DevX team has been instrumental in troubleshooting and fixing a wide range of issues, from broken windows builds and obscure LoadDLL errors to blst integration across Nix and Github CI. We've also initiated automatic uploads for release assets. Our efforts in streamlining complex CI setups have paid off, with some repositories like cardano-base experiencing significant reductions in CI complexity.

compiler upgrades​

After the support for 9.2 across our libraries, we have started working on 9.6 compabilitiy as well. This move brings us closer to the upstream compiler, facilitating the contribution of patches and enabling early detection of regressions. In addition, we're prioritizing compaining even stronger for better backwards compatibility.

CHaP (cardano-haskell-packages)​

We relocated the underlying tooling, (foliage), for CHaP into the IOG organization. Furthermore, we have introduced improved tooling to quickly add constraints to packages, better error reporting for add-revision and better hackage url compatibility to facilitate easer usage of CHaP.

GitHub Actions​

Our repository, input-output-hk/actions, now houses the necessary actions for installing pre-requisites to build Cardano projects using GitHub Actions. Leveraging the base and haskell install actions has allowed us to simplify workflows in the repos, focusing primarily on invoking cabal.

iohk-nix​

The iohk-nix repository has undergone a major revamp and now provides pre-built packages of the cryptographic libraries IOG utilizes for GitHub Runners. The key components we use, sodium, blst, and secp256k1, are also fixed to certain revisions within the iohk-nix repository.

haskell.nix​

Haskell.nix has been maintained and updated with the addition of GHC 9.6.2 and GHC 8.2.8. After discovering performance regression in the native bignum backend, we switched the default bignum backend to gmp.

devx​

The relatively new devx repository is where we experiment with a single nix development shell that aims to suffice for most use-cases at IOG. This initiative is expected to eliminate a number of CI failures related to project-build and shell interaction problems. The devx repository's readme has been updated to reflect its purpose and usage guidelines.

upstream tooling​

Our team remains committed to enhancing upstream tooling, with ongoing contributions to GHC, Cabal, and Nix.

High-level summary​

This week, the Hydra team worked on multiple fronts. They finished the investigation about the broken head on mainnet and re-opened their persistent head instance. The team also fixed the monthly report publication on their website and started sketching ideas and further improvements. Also, they are on the last mile to deliver a new feature which will allow parties to commit funds from extern wallets. Finally the team started to work on optimizing the performance on their benchmarks.

What did the team achieve this week​

• Finished investigation on broken head on mainnet #897 and re-opened it.
• Added support for externally committing regular utxo #887
• Fix monthly report publication on docs website and published the monthly report. Odd problems when publishing monthly report:
  • Make us think about if we should change something about the website #908
  • Open issue to docusaurus #9036
• Fixed a bug in the benchmark process #910
• Explored performance of the hydra-node{.verbatim} and identified a bottleneck.
• Timed transaction feature is being used by the auction project 🎉

What are the goals of next week​

• Complete performance analysis and start/plan improvements and provide regular benchmarks for Hydra #186
• Add hydra as tool to developer platform #872.
• Authenticate network messages #727.
• Complete journey for external commits using multiple script UTxOs #903
• Start implementing Option B for external commits #215.

High level summary​

We've made progress in all high level objectives

• CIP-1694 integration design
• UTxO-HD integration proof of concept
• schema migrations with a focus on speeding up frequent queries is part of release 13.1.1.2
• A new hire will help us with devx issues.

Lower level summary​

• We have an initial design for the Conway integration in db-sync
• The initial UTxO-HD integration resulted in a requested feuture that the consensus team is working on #88.
• Added two optional migrations that speed up querying UTxO #1416
• Work on improving the above feauture by adding tests and fixing tech debt #1429 #1419
• Fixed issue with ada_pots being wrong https://github.com/input-output-hk/cardano-db-sync/pull/1407/commits/1ad4521a5601e8e98dc06bba2826d6b2f9b4fcf4
• Added a process which fixes in place scripts with wrong CBOR serialisation. https://github.com/input-output-hk/cardano-db-sync/pull/1407/commits/8792f72e2485b320de26c02dacc4d504d377ba17
• Prepared a tag 13.1.1.2 which is tested and ready to release. It contains many of the above.

The team works on applied research and consulting in formal methods that is directly applicable to evidence based engineering in Core Tech and beyond.

High level summary​

This sprint the team has been working on giving and preparing public talks, also on preparing final versions of their ICE 2023 papers, and on analysing chain selection.

This sprint the team has been working on final revisions for the pre-proceedings versions of two ICE 2023 papers.

Details​

• Lambda Days keynote by Kevin Hammond: https://www.lambdadays.org/lambdadays2023

• Codebeam keynote by Peter Van Roy: https://www.youtube.com/watch?v=h8sE3Ai8Dsk; https://www.youtube.com/watch?v=h8sE3Ai8Dsk

• Preparing talks and final versions for two ICE 2023 contributions: https://www.discotec.org/2023/ice

• Analysing chain selection.

• The team is hiring a performance engineering intern - https://apply.workable.com/io-global/j/BFEC5B0AFF/

High level summary​

The SRE team continues work on CI and cardano environment improvements. Some notable recent improvements include: expanding the darwin CI cluster and providing new aarch64 builder support; adding bare metal bitte cluster capability with network overlay for high IOPS workload performance, such as explorer.

Lower level summary​

Bitte​

• Equinix bare metal capability was added to bitte: bitte-pull-194
• Update bitte nixpkgs, nix version, nomad driver, equinix lifecycle, misc bug fixes: bitte-pull-201

Bitte-cells​

• Improvements made for patroni cluster recovery of large databases: bitte-cells-branch

Cardano-graphql​

• Update cardano-graphql and nixos service: cardano-graphql-pull-815

Cardano-node​

• Update cardano-node to use the updated iohk-nix environments: cardano-node-pull-5241

Cardano-ops​

• Update cardano-graphql, adds monitoring improvements and misc bug fixes: cardano-ops-pull-411
• Work in progress on mixed legacy and p2p topology clusters in cardano-ops: cardano-ops-branch

Cardano-world​

• Add a declarative cardano-faucet dashboard: cardano-world-commit
• Remove a requirement for root user in Nomad docker images: cardano-world-pull-68
• Enable explorer in cardano-world for high IOPS metal access: cardano-world-pull-69
• Migrate explorer cluster to cardano-world: cardano-world-pull-75
• Migrate testnet metadata server to cardano-world, cleanup envs: cardano-world-pull-77
• Add a faucet restore delegation script to cardano-world for non-functioning delegated pools: cardano-world-pull-79
• Integrate node 8.x into cardano-world: cardano-world-pull-84
• Update cardano-world to use iohk-nix for node environments: cardano-world-pull-87
• Create a stakepool analysis query for cluster pool performance overview: cardano-world-commit
• Added an iptables fixup script for broken bridged networking nomad jobs: cardano-world-commit
• Improve cardano-world job automation with additional parameterization: cardano-world-pr-92
• Update preview network experimental config defaults: cardano-world-pr-93

Ci-ops​

• Update legacy darwin builders and buildkite agent for ci-world network overlay and monitoring: ci-ops-pull-108

Ci-world​

• Resolve ci-world linux buildkite agent cleanup bug: ci-world-commit
• Integrate new darwin builders into ci-world: ci-world-pull-22
• Migrate patroni HA db to it's own namespace in ci-world: ci-world-pull-24

Cicero​

• Implement a cicero webhook backoff with exponential decay plus jitter: cicero-pull-79

Iohk-nix​

• Update iohk-nix to be source of truth for node environments: iohk-nix-pull-534
• Update preview network experimental config defaults: iohk-nix-pull-544

Openziti​

• Add darwin intel and arm support to ziti-edge-tunnel nix modules: openziti-bins-pull-1