High level summary​

We have focused the last two weeks on CIP-1694, integration of the last several months of ledger work into consensus and node, and testing infrastrutcture that we will use in the conway ledger era.

Low level summary​

Conway rules​

We made a major update to the conway era so that the implementation is now in sync with the spec with respect to the ratification and enactment logic.

See the notes in pull-3291 for more details.

Constraint based generators​

This week we hit a major milestone in our efforts to build out better property based testing support for the main ledger properties. The new constraint based generators can now generate full ledger states with what is probably very close to the real constraints (if anything, it is under constrained). Next we will work on generating a transaction in the context of a ledger state, which would allow us to actually start using these generators for real tests.

See pull-3219.

Preparing a release, now with proper versioning​

After quite some time, we are ready to release a version of ledger that will work with a new version of consensus, using CHaPs.

See pull-3308.

Technical debt​

• We cleaned up the redeemer serialization code (to prevent future mistakes). See pull-3269.
• We added a note to the Alonzo spec, specifying that the transaction inputs are lexicographically ordered in the Plutus script context. See pull-3306.
• We fixed a problem with the address deserialiazation (we mistakenly fixed a bug in the Babbage era which cannot be fixed until Conway). See pull-3307.
• We fixed a problem with our nix build. See pull-3311.
• We fixed a problem with our NoThunks tests. See pull-3310.
• We improved our nightly tests. See pull-3316.

High level summary​

In the current sprint the networking team focused on fixing bugs and pushing forward implementation of eclipse evasion. We also found a bug in our simulation testing setup (in integration of test node). We also overviewed the work on extending handshake protocol which is delivered by Galois Inc.

We published ouroboros-network-0.4.0.1 and ouroboros-network-protocols-0.3.0.0 to CHaP.

We also fixed a bug in cardano-node which results in not being able to configure inbound connection limits, see PR #4902.

Together with Karl Knutsson (CF) we realised an issue in cardano-cli: it's validation of DNS names, IP address & ports when registering a stake pool should be more strict to protect against common mistakes which we identified on the chain. See issue #4929.

Detailed work log​

In PR #4385 we fixed two bugs in peer state actions. First one results in a busy loop if demotion from hot to warm times outs. This busy loop is eventually exited when mux exits (we reported this in our previous report). This fix made it to 1.35.6 release as well.

In addition the PR #4385 also fixes another bug which results in hot -> warm -> hot demotion / promotion busy loop.

The PR #4385 also fixed a bug in a node only used in simulation which resulted in not using chain-sync or block-fetch mini-protocols. In the review process, we realised that the header-body split in the simulated node requires further work (see PR #4419, which is under review).

The PR #4385 also extend our generators, which together with the above fix, cover the hot -> warm -> hot demotion / promotion busy loop.

In PR #4419 we introduce a ChainDB for our simulation node, which plays similar role to ChainDB in the ouroboros-consensus: a persistent (across simulated restarts) store of blocks which does chain selection. This ensures that the simulated node is using block-fetch to download blocks announced by chain-sync mini-protocol.

We also made progress with reviewing PR #4019 - peer sharing.

We also fixed issue #4370 - a connection manager test failure, see PR #4384.

High-level summary​

This week, the Hydra team continue closing many gaps and aligning the implementation with the specification. More over, they have groomed a plan to make Hydra Mainnet compatible. Also, they continue moving forward with the internal auditing and have published some auditing guidelines to receive contributions from the community.

What did the team achieve this week​

• Completed #452 (what is remaining in there)
• Aligned the Head protocol logic implementation with the specification
• Completed full minting policy implementation and spec #720
• Clarified message authentication with researchers
• Groomed what is left to do for Mainnet compatibility #713 and drafted a 0.10.0 version
• Added a tutorial by @perturbing to the website LINK
• Published auditing guidelines LINK
• Made hydra-node work for macos #746 and added support for building on aarch64 #673
• Met with a potential customer of Hydra for Payments

What are the goals of next week​

• Have the monthly review meeting incl. the report
• Have smoke tests fixed and running regularly
• Release 0.9.0 with updated scripts
• Redraw transaction graphs for specification

High level overview​

The Mithril team released a new 2306.0 distribution that implements minor fixes and updates and completes the wiring of the era activation markers reader into the signer and aggregator nodes. They also designed an event store in the aggregator and completed its implementation as a monitoring solution to provide figures for deployment rates of versions on signer nodes before activating an era switch. Additionally, the team created a dedicated command on their nodes’ CLI that allows generating and signing an era marker’s payload to be stored on the Cardano chain, as well as creating dynamic cases for the end-to-end tests they run on the CI.

Low level overview​

• Released the new distribution 2306.0
• Completed the epic that implements signer versions deployment monitoring #718:
  • Completed the implementation an event producer/consumer via channel #738
  • Completed the creation of a database and its configuration to save the events on the consumer side #740
  • Completed the creation of events and sending them on the channel on the producer side #741
  • Completed the creation of the signer registration event #742
  • Completed the creation a query to extract the node versions stakes distribution #743
• Worked on the epic that implements eras behavior switch #707:
  • Completed the loading of era reader adapters from config in the signer and the aggregator #732
  • Completed the implementation of an era cli command in the aggregator #755
  • Completed the implementation of a dynamic matrix of cases in CI end to end tests #760
• Fixed some bugs:
  • Fixed the unsupported unixepoch() function #757
  • Fixed the problem that prevented some signers from signing on the testing-preview network #730
  • Update SQLite version to 3.40 in aggregator infrastructure #765

• SECP benchmarking: we concluded our benchmarking runs and analyses of the new SECP primitives for the Valentine hard-fork.
• Release benchmarking: we performed a round of benchmarks for the 1.35.6 release.
• UTxO-HD benchmarking: we performed first runs for UTxO-HD and are currently refining the benchmarking setup.
• New tracing: for better accessibility, the new tracing system is being outfitted with introspective capabilities.
• Infrastructure: with the Nomad cloud workbench backend we were able to perform our first test cluster runs successfully on SRE infrastructure.
• Infrastructure: the initial NixOps workbench backend has been completed; a PR containing this work, along with many quality-of-life improvements of our tooling, got merged.

Performance

SECP​

1. For SECP, we settled on a fixed tx count per block, while simultaneously spending as much as possible of the block budget. Thus we were able to minimize the impact of per-SC-call overhead.
2. The final runs were performed with various fractions, e.g. half, of the current block budget to ascertain how these workloads would fare compared to a value-only run.
3. The SECP machinery and profiles are currently being generalized into an approach to aim for very specific aspects of a smart contract for benchmarking.

UTxO-HD​

1. After analyzing initial UTxO-HD runs, it turned out that mempool snapshotting had to be throttled for benchmarking; it affects a lock that UTxO-HD had to introduce into the forging loop.
2. We're currently adapting the benchmark setup to that, and will then perform a new combination of baseline and UTxO-HD runs.

1.35.6 release​

Benchmarking the 1.35.6 release candidate could attest to a perfectly clean bill of health.

Tracing

Work on the new tracing system's introspective capabilites is ongoing: Immediate use cases of the new API include being able to statically validate generated tracer documentation, as well as providing information of a specific tracing setup in the node via traces themselves. These features will make the new system both more robust, and more accessible.

Infrastructure

Nomad backend​

1. Work on the cloud deployment capability of the Nomad workbench backend continued; for testing we can automate multiple Nomad clients.
2. Locality assumptions were removed and job monitoring was refactored.
3. To facilitate directly-executable derivations, Nomad Job specification files are now self contained with GitHub references and configs needed to run a cluster.
4. We're currently evaluating different options for genesis distribution in said cluster.

NixOps backend​

The NixOps workbench backend has reached an initial functional stage. Consequently, the relevant PR was merged. It also contained many improvements to our analysis tooling, as well as a structural overhaul of workbench itself. We consider this an important step of future-proofing our benchmarking machinery.