Skip to main content

· 3 min read
Carlos LopezDeLara

2023-08-30 - 2023-09-12

High level summary​

Release of Cardano-node 8.3.0-pre for SanchoNet. Adds DReps to the Conway governance rules. Allows registration of DReps, delegation to DReps and voting on constitution action with DReps and SPOs. This release completes phase 2 of SanchoNet feature roll out plan

CLI continues on its migration to a era-based top-level commands. This sprint we moved text-view, key, query, genesis, node, stake-address, transaction and address into this new command structure. This is the initial migration, there is still some clean-up to do on future sprints.

We are moving the drep delegation-certificate to the stake-address command so delegating to a drep is done via stake-addres vote-delegation-certificate. Along the same lines, we are renaming stake-address delegation-certificate (delagating to a stake pool) to stake-address stake-delegation-certificate to distinguish between stake delegation to a pool and votes delegation to a drep. On top of that, there is a new type of delegation certificate to delegate (stake) to a stake pool and (votes) to a drep simultaneosuly stake-address stake-and-vote-delegation-certificate. Note that change is not included on 8.3 but will come on 8.4. Removing --conway-era flag from all the transaction sub-commands.

Update description fields in delegation certificates from Stake Address Delegation Certificate to respectively (Conway onwards):

  • Stake Delegation Certificate
  • Vote Delegation Certificate
  • Stake and Vote Delegation Certificate

cardano-cli​

cardano-api​

cardano-node​

cardano-testnet​

docs​

CI & project maintenance​

· 2 min read
Sebastian Nagel

High-level summary​

This week, the Hydra team improved the commit process with support for inline datums. They simplified the chain layer by refactoring how the chain state is persisted. Updates to the Hydra tutorial were completed, including CI workflows to keep it up-to-date. The team engaged in discussions with researchers about incremental commits & decommits, and drafted an architectural decision record for a resource-based API. They also explored simpler solutions for the upcoming network resilience feature.

What did the team achieve this week​

  • Support InlineScriptDatum in commit #1043
  • Refactored chain state persistency #1049
  • Completed tutorial updates (by adding CI and some cleanup) #997
  • Intersect Open Source committee meeting
  • Experimented with various models to better express the \"Network resilience problem\" to find a KISS solution for #188
  • Drafted and discussed incremental de-/commit features within team and with researchers #199 and #1057
  • Created ADR for a resource based overhaul of the API #1028

What are the goals of next week​

  • Meetup of Hydra & Mithril contributors in Nantes, France
  • Integrated cardano-api 8.15 to have GHC 9.6 support
  • A clear design for incremental commits (decommits are already understood)
  • An ADR for improving tx construction & observation to not need chain state

· 2 min read
Jean-Philippe Raynaud

High level overview​

The Mithril team has released a new distribution 2335.0 which brings enhancements to the client: support for simultaneous downloading and unpacking of the snapshot archives, support for one line command restoration of the latest snapshot, and availability of machine readable progress information.

They have also worked on snapshots enhancements: implementation of the zstandard compression to produce smaller archives, and providing the version of the Cardano node used to create the archive in the snapshot artifacts. They have completed the second phase of the implementation of the stress test tool to benchamrk the aggregator. Additionally, they have kept working on refactoring and standardizing errors in the Mithril nodes.

Finally, they have worked on adding Cloudflare protection to the infrastructure, and they have fixed some bugs that occurred during restoration of a snapshot by a client.

Low level overview​

  • Completed the issue Use 'zstandard' compression for snapshot archives #876
  • Completed the issue Design & implement stress test tool for aggregator - phase 2 #1155
  • Completed the issue Provide progress information with '--json' option in Client #1095
  • Completed the issue Make client download and extract the archive simultaneously #1115
  • Completed the issue Add aggregator Cardano node version in snapshot artifact #948
  • Completed the issue Make Cardano node version part of the Mithril network configuration #947
  • Completed the issue Post deployment 'mainnet' infrastructure #1091
  • Worked on the issue Errors refactoring #798
  • Worked on the issue Add Cloudflare protection of infrastructure #986
  • Completed the issue Failed to restore recent snapshot with strange error #1160
  • Completed the issue Update 'ed25519-dalek' to '2.0.0' #1188
  • Completed the issue Add target networks blocks in release notes #1151
  • Completed the issue Fix end to end 'wait_for_the_expected_time' test #1191
  • Completed the issue Fix docs 'git checkout' command #1174

· 3 min read
Michael Karg

High level summary​

  • Benchmarking: We've performed and analysed feature benchmarks for both UTxO-HD and the current P2P stack.
  • Infrastructure: Various improvements of our analysis pipeline have been merged to master, supporting safe log truncation.
  • Tracing: Namespace consistency checks have been merged to master along with a curated configuration for benchmarking.
  • Nomad backend: We're productively using the new backend to measure new vs. legacy tracing system, adding many quality-of-life improvements.

Low level overview​

Benchmarking​

We've completed various runs and analyses targeting two distinct features of the node: UTxO-HD and Peer2Peer.

With our UTxO-HD benchmark we could clearly localize one point where this new way of maintaining ledger state is still costly, but at the same time confirm that in basically all other aspects UTxO-HD makes no difference in performance.

The Peer2Peer benchmarks focused on the effects that enabling this feature on a block producing node has on propagation times, as well as scrutinized a proposed change to the Peer2Peer network stack.

Infrastructure​

As a result of optimizing in-memory representation of log objects, which are constructed from a node's traces, we can now analyse runs that last longer in total. For runs that exceed their expected duration, analysis now supports a truncation operation that keeps the interdependencies of block events intact.

Truncation might happen at a slightly different point in time - and therefore in its log object stream - for each node in the cluster. An additional step validating the block hash timeline of the cluster has been implemented for the pipeline. It provides early feedback on whether a specific truncation will lead to a valid full analysis, which requires much more time.

Tracing​

Consistency checking of namespace implementation and configuration when using the new system has been completed. This feature enables feedback on when tracer implementation details in some component might have changed. It's also able to detect when a configuration used for operating a cardano-node shows inconsistencies with the namespaces the system provides - and hence needs attention.

Furthermore, we've created a fine-grained configuration of the new system that caters to benchmarking's need of very many detailed trace messages. It's aimed at mirroring the same amount of trace messages, and information, we're seeing from our usage of the legacy system; an important step in making benchmarks between systems comparable.

Nomad backend​

The new backend is currently being used for further validation with regard to the existing cluster. Additionally, we're using it in production mode to comparatively benchmark both tracing systems after merging past month's optimizations - which is the first real-life application of the nomad cluster. Hands-on experience in that phase translates into many small improvements which can be immediately applied to enhance user experience for the new backend.

· 3 min read
Damian Nadales

High level summary​

We were able to successfully run the system-level benchmarks for the UTxO-HD implementation, for the first time. There was an important regression in block forging performance that will have to be addressed before UTxO-HD is released. We also revisited the implementation of our query processing logic, which was needed to address the performance regression found in the query-by-address command. The preliminary performance results show that now the performance of this query is on-par with the Cardano baseline version, but we need further confirmation. On the Genesis front, we presented the grinding-aware safety argument for the proposed historical Cardano Genesis windows to the IOG Researchers. The Consensus release engineer finished his rotation: version 8.3.0-pre of cardano-node is releasing 2023 September 5.

UTxO-HD​

  • We ran the first successful system-level benchmarks for UTxO-HD (see #203) using the in-memory backend.
    • We observed a factor 12 regression in the forging performance, which we will have to address. There are strong indications that the regression is due to the backing store accesses that take place when taking a mempool snapshot.
    • After the mempool regression is fixed the benchmarks need to be ran again.
    • System-level UTxO-HD benchmarks with the LMDB are still pending.
  • UTxO-HD will eventually be necessary due to the growth of the UTxO set and other ledger state structures that live in memory at the moment. However, we are trying a strategy by which we could preserve the baseline performance of the node, in case SPOs and other node users are not ready to migrate yet (see #344).
  • We implemented a new way of processing queries at the hard-fork block level, which resolves the performance regression observed in GetUTxOByAddress (see this comment). Preliminary results are promising.
  • Regarding the roll out plan, UTxO-HD requires a significant change in the Consensus codebase. Even though we might be able to hide any potential performance impact in the node by keeping all data in memory (#344), the Consensus component was significantly changed, so we might have to postpone releasing this feature to mitigate any risks of conflicting with the implementation of CIP-1694 and release of Conway.

Tech debt​

  • We added tests that Consensus emits valid CBOR (#3099). This helped us detect a couple of serialization bugs. The tests still need to be merged into the main branch (#323).

Support​

  • Nick Frisby finished his release engineer rotation; cardano-node 8.3.0-pre is releasing 2023 September 5.
  • We helped to investigate a protocol version bug in Sanchonet (see #3491).
  • We started to implement the Network interface for bootstrap peer functionality, from which Genesis will benefit as well (see #91.