Skip to main content

· One min read
Noon van der Silk

High-level summary

Returning from a well-earned break the team has merged the major part of the work on incremental commits and is now seeking feedback from the community on this feature. We continue to work on the final changes (in the spec) and continue testing the feature ourselves. We carry on with our work on the hydra-explorer and custom ledger investigations, as wel as getting into some planning for the new year.

What did the team achieve?

  • Merged the bulk of the incremental commits work! #1715
  • Progress on custom ledger experiment #1742
  • Progress on Hydra explorer supporting multiple versions #1282
  • Docusaurus upgrade #1768
  • Staying up-to-date with cardano api #1760 and #1772
  • Various cleanups and debugging improvements: #1755, #1749, #1767, #1762, #1774, #1776.

What's next?

  • Final work on incremental commits #199
  • Finish Hydra explorer supporting multiple versions #1282
  • Finish custom ledger experiment #1742
  • Plan the 0.20.0 release
  • Continue support Hydra Doom

· 2 min read
Jean-Philippe Raynaud

High level overview

The Mithril team released the new distribution 2450.0, which offers stable support for Cardano node v.10.1.3 and includes bug fixes and performance improvements.

The team also continued implementing the incremental certification of the Cardano database: they kept working on creating and synchronizing the artifacts, implemented the artifact routes, and started working on the digest route for the incremental Cardano database in the aggregator REST API.

Finally, they completed the experimental cache for certificate verification in the WASM client, implemented enhancements to the CI, and fixed a bug with identifier collisions in the incremental Cardano database artifacts.

Low level overview

  • Released the new distribution 2450.0
  • Published a dev blog post about the Era switch to Pythagoras
  • Published a dev blog post about the Distribution 2450 is now available
  • Completed the issue Implement artifact routes for incremental Cardano database in aggregator REST API #2152
  • Completed the issue Make devnet bootstrap errors retryable #2176
  • Completed the issue Support stable and unstable versions of the explorer #2172
  • Completed the issue Handle cache for certificate verification in WASM client #1484
  • Completed the issue Run nightly Aggregator Stress Test and Client Multi Platform workflows #2177
  • Completed the issue Nightly backward compatibility tests failing #2193
  • Completed the issue CardanoDatabase artifact identifiers have collisions #2197
  • Completed the issue Release 2450 distribution #2124
  • Worked on the issue Implement digests route for incremental Cardano Database in aggregator REST API #2174
  • Worked on the issue Design mithril-common split & re-organization in repository #2175
  • Worked on the issue Implement artifacts builder for incremental Cardano database #2151
  • Worked on the issue Activate Pythagoras Mithril era #2034

· One min read
Jordan Millar

2024-12-18T00:00:00Z - 2025-01-01T00:00:00Z

High level summary

  • SPO Stake Distribution: Added functionality to query spo-stake-distribution that reveals DRep delegation choices of stake pool rewards accounts.
  • API Release: Published cardano-api-10.5.0.0
  • API Exports: Added the export of genTxOutByron to support external tools that need to work with Byron-era transaction outputs.

cardano-cli

cardano-api

cardano-node

  • None

cardano-testnet

  • None

docs

CI & project maintenance

  • None

· 5 min read
Michael Karg

High level summary

  • Benchmarking: Finalized voting benchmarks on Node 10.0; workload implementation being generalized to be mergeable.
  • Development: Database-backend for our analysis tool locli merged; several metrics improvements with new tracing.
  • Tracing: C library for trace forwarding started; documentation improved; timing issue in forwarder fixed.

Low level overview

Benchmarking

The voting benchmarks have now finished. The exact implementation of how the voting workload is set up and submitted has been finalized and is currently being prepared for merging into master. This will add those benchmarks to the repertoire we can run on any future node version, and track potential performance changes of voting over time.

The setup allows us to add voting as an additional workload on top of the existing release benchmarking workloads - typically "value-only" and "Plutus loop". The value workload operates at 12TPS and always results in full blocks; we can draw a straight line comparison when a certain, constant percentage of each blocks is filled with vote transactions. The Plutus workload however is throttled by spending the block execution budget, and not so much by transaction size and TPS - contrary to voting submissions. This results in a large variance in block size that the network produces, and restricting analysis to the blocks that are actually comparable to each other greatly reduces sample size.

This means that in practice, we've found "voting on top of value-only" to represent the performance implications of voting most accurately. This workload will serve as a base for comparison over time, and will be run selectively on new versions, whenever the proposal / voting feature of the Conway ledger is touched.

As a conclusion to those benchmarks we've ascertained that:

  1. there is a performance cost to voting, vote tallying and proposal enactment
  2. on the system level, this cost is very reasonable and poses no operational risk
  3. on the service level, processing an individual voting transaction is even slightly cheaper performance-wise than a transaction consuming and creating multiple UTxO entries

Development

The analysis and reporting tool, locli ("LogObject CLI") now comes equipped with a database-backed persistence layer. This new backend has been validated by using it to re-analyse past benchmarks. Performance workbench integration has been completed, and by means of a new envvar this backend can be enabled for use in automations. It currently co-exists in locli with the default file system based persistence backend.

Apart from opening up raw benchmarking data to the full power of SQL queries, or quickly marshalling it into another format to feed into other applications, the new storage backend has considerable advantages regarding execution speed and resource usage. It both requires less RAM (around 30% less) during runtime, and less disk space - about 90% less! Standard analysis of a cluster run can now be performed in less than an hour, whereas it took around 2h before.

Currently, we're working on implementing parametrizable quick queries of that raw data - complete with adding plotting capabilities to locli. The queries are meant to easily extract and visualize very specific correlations that are not part of standard analysis, catering to the oftentimes investigative nature of performance analysis.

Furthermore, The new tracing system now provides direct insight into the chain tip's hash, exposing tipBlockHash, tipBlockParentHash and tipBlockIssuerVerificationKeyHash both as trace messages and metrics. Additionally, we've merged a fix for issue cardano-node#5751: the metric forging_enabled now correctly also observes the presence of the CLI option --non-producing-node.

Tracing

The new tracing system allows for trace and metrics forwarding from some process to cardano-tracer. For any Haskell application, the forwarder package can easily be included as a library. For applications written in other programming languages, we've decided a small, self-contained C library that handles forwarding is a viable way to provide this functionality to a much wider range of ecosystems. The C library will implement protocol handshake and possibly muxing, the forwarder protocol messages being used, and CBOR-based encodings of trace messages and metrics - which only exists in Haskell currently. We've just started the prototype.

We've been working hard on updating and improving the documentation for the new tracing system on https://developers.cardano.org (not merged yet). The aim was to provide a quick start guide to "just get it set up and running", without presupposing any knowledge of tracing, or Haskell. Moreover, for users coming from the legacy tracing system, we wanted to highlight the key differences between systems - and possibly different assumptions when operating them.

Last not least, we caught a very interesting timing issue in the forwarder: each service connected to cardano-tracer bears both an internal and external name for the connection (both unique), where the external name is chosen by the service itself. Upon forwarder initialization, so called data points are set up within the service, into which data can then be traced (such as that external name), and which are actively polled / queried by cardano-tracer. As these are all concurrent events, the external name wasn't yet available in the data point, if initialization of forwarding happened "too fast". Once located, fixing this was trivial by enforcing a relative ordering of concurrent events just during initialization.

Happy New Year!

It's been an amazing year for the Performance & Tracing team. We're proud to have contributed to Cardano's transition into the age of Voltaire, and reliably safeguarded performance of the Cardano network - and to have finalized our new tracing system. A huge thanks to all those who've been helpful, supportive - and who've presented us with new ideas and challanges.

Have a Happy New Year 2025!

· 2 min read
Marcin Szamotulski

Overview of sprint 77

Initiator only mode for local roots

We implemented the initiator-only mode for local roots peers described in ouroboros-network#5020. This feature will be available in cardano-node-10.3 (cardano-node#6055).

One will be able to specify diffusionMode (either InitiatorOnly or IntiatorAndRespoinder, the latter is the default) for all local roots in a given local roots group, e.g.

{ "localRoots":
[ { "accessPoints":
[ { "address": "10.0.0.1"
, "port": 3001
}
]
, "advertise": false
, "diffusionMode": "InitiatorOnly"
, "warmValency": 1
, "hotValency": 1
}
, { "accessPoints":
[ { "address": "10.0.0.2"
, "port": 3001
}
]
, "advertise": true
, "diffusionMode": "InititiatorAndResponder"
, "warmValency": 1
, "hotValency": 1
}
]
, "publicRoots": []
, "useLedgerAfterSlot": -1
}

As part of ouroboros-network#5020 we had to change how connections are identified in the simulated testnet environment. We exposed ConnStateIdSupply through P2P interfaces, which allows us to use a global ConnStateIdSupply for all nodes in the simulation. This way, ConnStateId is unique for each connection in the simulation. See ouroboros-network#5026.

Support systems with multiple IP addresses

We merged ouroboros-network#5017 which allows to run cardano-node on systems with multiple network interfaces or a single interface with multiple IP addresses.

Reusable diffusion

We were working on early integration of reusable diffusion work stream with ouroboros-consensus & cardano-node. Reusable diffusion will allow us to support both cardano-node and mithril-node in the future. We are pleased to say that we are running a cardano-node that uses refactored diffusion. See:

Tx-Submission Logic

We had a discussion with the consensus team (Karl Knutsson CF, Nick Frisby Tweag) on network requirements for tx-mempool. See:

SRV record support

We continued working on the SRV support, see:

Technical Debt

We renamed some of the test modules to be more consistent across various network components, see ouroboros-network#5028.