High level summary​

These were our main areas of focus last period on the Ledger project:

Maintenance​

We worked really hard in order to prepare Ledger for the cardano-node-10.7.0 release that had all of the bug fixes and features needed for the upcoming van Rossem Hard Fork into protocol version 11. Furthermore, we prepared Ledger for a subsequent cardano-node-10.7.1 release that had some of the performance issues resolved, which were identified in the Ledger component of 10.7.0 release.

We've also identified and fixed a few bugs, most of which were in CDDL specification. This was largely thanks to the recently integrated tool AntiGen, that we developed specifically for creating negative property tests for Ledger serialization.

Besides bug fixes, we've added quite a few nice optimizations, which do result in noticeable speed up in chain validation and replay.

Some very nice improvements were implemented for the Ledger API used by Consensus, which will result in clearer and safer boundaries between those two components.

CIP-118 - Nested Transactions:​

Two more major Ledger rules UTXO and UTXOW were defined for processing transactions in the Dijkstra era that now take sub-transactions into consideration.

CIP-165 - Canonical Ledger State:​

Fixes were applied to canonical ledger state representation for the Constitutional Committee

Low level summary​

Features​

• PR-5654 - Normalize StakeRefPtr addresses better and improve UTxO decoding
• PR-5647 - Change B constructor of Metadatum from ByteString to ByteArray
• PR-5655 - Add ToJSON/FromJSON instances for NonEmptyMap and NonEmptySet
• PR-5630 - Remove distinct_bytes hack, make CDDL reflect reality in previous eras
• PR-5560 - *EraBlockHeader typeclasses deprecating BHeaderView
• PR-5658 - Optimize Text byte length check in metadata and other places
• PR-5632 - Forecast API
• PR-5671 - Remove NoThunks instances for predicate failures and ContextError types
• PR-5635 - ScriptPurpose aware memoization of TxInfo
• PR-5676 - Remove state update from UTXOS
• PR-5641 - Avoid unnecessary computation of PlutusWithContext
• PR-5672 - Make ContextError constructors from Alonzo to Dijkstra era lazy
• PR-5614 - SUBUTXO implementation
• PR-5668 - SUBUTXOW fixes
• PR-5704 - Backport: revert ConwayAccountState type unrolling
• PR-5697 - Remove 5 redundant NativeScript era ~ Timelock era constraints (#5319)
• PR-5665 - Fix metadatum decoder and chain_code CDDL
• PR-5681 - Add/refactor committee-related canonical namespaces
• PR-5725 - Revert decoders to a more efficient implementation
• PR-5716 - Move applyTick into its own ApplyTick typeclass
• PR-5719 - Add protocol version validation to createInitialState
• PR-5729 - Fix ConwayAccountState overhead
• PR-5727 - Add storable instances

Testing​

• PR-5656 - Translate all transaction building examples in our test files to use Lens interface.
• PR-5609 - Adequate conformance to Version restriction
• PR-5667 - Fix Proposal generation to generate HardForkInitiation with bounded major version
• PR-5695 - Update formal-ledger-specifications and re-enable tests
• PR-5529 - Add simpleRewards reproducer from cardano-db-sync and migrate TwoPools example to an ImpTest
• PR-5715 - Bump cuddle to 1.5.0.0

Infrastructure and releasing​

• PR-5653 - Add cabal-gild and shellcheck as part of the pre-commit hook in our Nix devShell.
• PR-5648 - Changelog for cardano-node-10.7
• PR-5657 - Migrate scripts to cardano-ledger-release-tool
• PR-5599 - Enable ghc 9.14 to be used for tests in CI and locally
• PR-5677 - Bump requests from 2.32.4 to 2.33.0 in /doc
• PR-5675 - Add nixfmt in our Github CI checks
• PR-5670 - Add CHANGELOG and cabal file updater skill
• PR-5691 - Bump pygments from 2.19.1 to 2.20.0 in /doc
• PR-5702 - Update contribution policy to note how contributions should be vetted
• PR-5703 - Add a CI workflow for checking nix hashes
• PR-5708 - Bump the actions group with 3 updates
• PR-5709 - Update slack-github-action configuration for v2 onward
• PR-5712 - Fix missed version bump for allegra package
• PR-5713 - Stop running bump-changelogs on release branches
• PR-5666 - Refactor transaction and transaction body of examples in Test.CardanoLedger.<era>.Examples.hs.
• PR-5717 - claude: improve changelog update skill
• PR-5710 - Update microlens dependency
• PR-5726 - Update ghc-9.12 on CI and Nix to ghc-9.12.4, and cabal-3.16 to cabal-3.16.1.0
• PR-5728 - Backport of fixes for cardano-node-10.7.1
• PR-5741 - Fix nix build failure in plutus-preprocessor
• PR-5721 - Leios: Make the (leios enhanced) block decoding backwards compatible

High level overview​

This week, the Mithril team completed several SNARK-related tasks. They created a new module for the recursive SNARK circuit in the STM library, added support for test mode, refactored the bytes codec for forward and backward compatibility, synchronized updates from the non-recursive circuit prototype, prepared a testing plan for the IVC circuit, enhanced witness preparation for the non-recursive prover, and added golden tests for the recursive SNARK circuit prototype.

They also completed partial block range support, security parameter offset signing, and the WASM client implementation for Cardano blocks and transactions. In addition, they continued work on the explorer update and on documenting Cardano block and transaction signatures on the website. The team also progressed the upgrade to Cardano node v.10.7 and completed DMQ node updates to 0.4.1.0 and 0.4.2.0.

Finally, they completed cross-compiled Windows builds from Linux using Nix, fixed node build failures in Hydra CI, and investigated a SNARK aggregator certification stoppage on the dev network.

Low level overview​

Features​

• Completed the issue Refactor bytes codec in STM library for forward/backward compatibility #3065
• Completed the issue Support test mode for the recursive circuit #2984
• Completed the issue Create recursive SNARK circuit new module in STM #3123
• Completed the issue Synchronize updates of non recursive circuit prototype to STM library #3121
• Completed the issue Support partial block range in Cardano blocks and transactions #3099
• Completed the issue Sign security parameter offset in ProtocolMessage for Blocks and Transactions #3098
• Completed the issue Implement Cardano Blocks and Transactions in WASM client #3078
• Completed the issue Prepare Testing plan for IVC circuit #3124
• Completed the issue Implement a Cardano bundle Docker image #3189
• Worked on the issue Prepare production readiness for circuit keys and trusted setup in STM #3165
• Worked on the issue Enhance preparation of witness for the non-recursive prover in STM #3178
• Worked on the issue Add golden tests for recursive SNARK circuit prototype #3125
• Worked on the issue Update explorer for Cardano Blocks and Transactions #3079

Protocol maintenance​

• Completed the issue Update DMQ node to 0.4.1.0 #3114
• Completed the issue Update DMQ node to 0.4.2.0 #3196
• Completed the issue Cross-compile Windows builds from Linux using Nix #1886
• Completed the issue Mithril node build fails in Hydra CI #3161
• Worked on the issue Upgrade to Cardano 10.7 #2894
• Worked on the issue Document Cardano Blocks and Transactions signature in website #3101
• Worked on the issue Update client test in CI to check blocks and transactions #3159
• Worked on the issue SNARK aggregator certification stopped on dev-follower-preview #3183

High level overview​

This week, the Mithril team completed the full review of the recursive SNARK circuit prototype, the impact assessment of SNARK on Mithril protocol security, and support for multiple proof systems in the STM library. They also completed SNARK proof generation and verification in end-to-end tests. They began work on the recursive SNARK circuit, including creating a new module in the STM library, and worked on a testing plan for the IVC circuit and production readiness for circuit keys and trusted setup. They also supported test mode for the recursive circuit and continued refactoring the STM library's byte codec.

Additionally, they completed the client CLI implementation for Cardano blocks and transactions and made progress on partial block range support, security parameter offset signing, and the explorer update for Cardano blocks and transactions.

Finally, the team completed the reqwest upgrade to 0.13 and continued work on the Cardano node upgrade to 10.7 and the DMQ node update to 0.4.1.0.

Low level overview​

Features​

• Completed the issue Full review of recursive SNARK circuit prototype #2982
• Completed the issue Impact of SNARK on Mithril protocol security #2803
• Completed the issue Support Multiple proof systems in STM #2550
• Completed the issue Implement SNARK proof generation and verification in end to end tests #3107
• Completed the issue Implement Cardano Blocks and Transactions in client CLI #3032
• Worked on the issue Refactor bytes codec in STM library for forward/backward compatibility #3065
• Worked on the issue Support test mode for the recursive circuit #2984
• Worked on the issue Create recursive SNARK circuit new module in STM #3123
• Worked on the issue Prepare Testing plan for IVC circuit #3124
• Worked on the issue Prepare production readiness for circuit keys and trusted setup in STM #3165
• Worked on the issue Support partial block range in Cardano blocks and transactions #3099
• Worked on the issue Sign security parameter offset in ProtocolMessage for Blocks and Transactions #3098
• Worked on the issue Update explorer for Cardano Blocks and Transactions #3079

Protocol maintenance​

• Completed the issue Upgrade reqwest to 0.13 #3033
• Worked on the issue Upgrade to Cardano 10.7 #2894
• Worked on the issue Update DMQ node to 0.4.1.0 #3114

High level summary​

The Plutus team has recently made a number of significant improvements to Plinth, including:

• Compiler improvements:
  • Added a type checker plugin that preserves source locations, resulting in significantly clearer error messages (#7640).
  • Added a second type checker plugin to detect unsupported Haskell features, further improving error reporting (#7659).
  • Added a driver plugin that automatically sets the required compiler flags and enables the Strict extension (#7687).
• Language improvements:
  • AsData now generates a destructor function for the data type (#7664). For matching on sum types, the destructor function is more efficient than the pattern synonyms.

The Plinth user guide will be updated soon. In the meantime, please refer to the linked PR descriptions for further details.

Additionally, a new UPLC optimization has been implemented: applications with three or more arguments are now transformed into case-constr form.

Low level summary​

Key Pull Requests Merged​

• Improve compile errors, and add Plinth.Plugin that preserves more source locations
• Plinth: improve error reporting for unsupported Haskell features
• Generate a destructor function from AsData
• Drop redundant unsafeCaseList calls produced by AsData
• Automatically set required flags and Strict extension in the plugin

High level summary​

• Ouroboros Leios (Treasury Funding Initiative 4: Ouroboros Leios Implementation):
  • Implemented Leios Endorser Block (EB) inclusion in the consensus layer, adding EB announcement and certification tracking to blocks, along with a SQLite-based backend for querying EBs and certificates (#1921).
• UTXO-HD (Treasury Funding Initiative 10: LSM including UTXO-HD):
  • Refactored database initialization by modifying mkOpenState in the ImmutableDB and VolatileDB (#1917).
  • Improved performance by caching transaction differences on first execution, addressing a hotspot observed during transaction revalidation on stressed nodes (#1954).
• Releases and integration (Treasury Funding Initiative 17: Maintenance and Support):
  • Released ouroboros-consensus-2.0.0.0 (#1947) and ouroboros-consensus-3.0.0.0 (#1964).
  • Integrated ouroboros-network-1.1.* (#1943).
  • Integrated the latest consensus packages into cardano-node for the upcoming Node 10.7 release (cardano-node#6402).
• Testing and documentation (Treasury Funding Initiative 17: Maintenance and Support):
  • Fixed a flaky ChainDB StateMachine test related to iterators (#1948).
  • Replaced the consensus documentation's Introduction page with a comprehensive System Overview, including a C4 Context diagram and clarifications on code organization and era evolution (#1950).