High level overview​

This week, the Mithril team completed the missing IVC off-circuit checks, the property-based tests for the IVC pure helpers, the caching of the circuit keys with the reuse of the SNARK prover setups, and the enhancements of the end-to-end tests for the IVC SNARK. They continued implementing the circuit verification key whitelist/revoke mechanism and supporting an IVC follower aggregator joining the network at any time. They also started implementing the Relation trait for the IVC circuit and adding a Proof of Bound Possession for the SNARK verification keys.

The team completed the IPFS support in the end-to-end tests, the test infrastructure for IPFS nodes, and the summary of the IPFS PoC. They also started adding a certificate chain cache committed after chain verification in the client library.

Finally, they kept upgrading to Cardano node 11.1, reduced the aggregator compile time, and kept investigating the blocks and transactions verification failure on the testing-preview network.

Low level overview​

Features​

• Completed the issue Add missing IVC off-circuit checks #3381
• Completed the issue Add property-based tests for the IVC pure helpers #3469
• Completed the issue Cache the circuit keys and reuse the SNARK prover setups #3424
• Completed the issue Enhance end-to-end tests for IVC SNARK #3390
• Completed the issue Implement IPFS in the end to end tests #3460
• Completed the issue Prepare summary for IPFS PoC #3519
• Worked on the issue Implement circuit verification key whitelist/revoke mechanism #3148
• Worked on the issue Support an IVC follower aggregator joining the network at any time #3446
• Worked on the issue Implement Relation trait for the IVC circuit #3428
• Worked on the issue Add a SNARK verification key Proof of Bound Possession #3537
• Worked on the issue Add a certificate chain cache committed after chain verification in the client library #3520

Protocol maintenance​

• Completed the issue Implement test infrastructure for IPFS nodes #3461
• Completed the issue Reduce the aggregator compile time #3538
• Completed the issue Publication to doc.rs failed for distribution 2630 #3507
• Worked on the issue Upgrade to Cardano 11.1 #3346
• Worked on the issue Blocks and transaction verification fails on testing-preview #3526

High level overview​

This week, the Mithril team completed the behavior tests for the IVC wiring, the harmonization of the recursive and non-recursive SNARK implementations in STM, and the STM README with recursive and non-recursive examples. They continued adding the missing IVC off-circuit checks, implementing the circuit verification key whitelist/revoke mechanism, and supporting an IVC follower aggregator joining the network at any time. They also started adding property-based tests for the IVC pure helpers.

The team completed the adaptation of the documentation for the on-chain configuration parameters. They also completed the IPFS file downloader of the client and started implementing the IPFS support in the end-to-end tests.

Finally, they kept upgrading to Cardano node 11.1, started fixing the failed publication of the distribution 2630 to docs.rs, and started investigating the blocks and transactions verification failure on the testing-preview network.

Low level overview​

Features​

• Completed the issue Add behavior tests for the IVC wiring #3465
• Completed the issue Harmonize the recursive and non-recursive SNARK implementations in STM #3421
• Completed the issue Adapt documentation for on-chain configuration parameters #3398
• Completed the issue Implement an IpfsFileDownloader in the client #3459
• Completed the issue Update STM README with recursive/non-recursive examples #3385
• Worked on the issue Add missing IVC off-circuit checks #3381
• Worked on the issue Implement circuit verification key whitelist/revoke mechanism #3148
• Worked on the issue Support an IVC follower aggregator joining the network at any time #3446
• Worked on the issue Add property-based tests for the IVC pure helpers #3469
• Worked on the issue Implement IPFS in the end to end tests #3460

Protocol maintenance​

• Worked on the issue Upgrade to Cardano 11.1 #3346
• Worked on the issue Publication to doc.rs failed for distribution 2630 #3507
• Worked on the issue Blocks and transaction verification fails on testing-preview #3526

High level summary​

Casing on Data has been merged: case on a Data.Constr value now dispatches directly on the constructor tag, which, combined with the recently added dropList, significantly reduces the cost of consuming Data-encoded values. This is a Plutus V4 feature, and we are also working on specifying casing on built-in types in the Plutus Core specification.

Work on the Value builtins (CIP-0168) continues: the keepPolicies and dropPolicies builtins have been added, the cost model for policies has been merged, and the cost models for keepPolicies and dropPolicies are in progress. Like the other new builtins, these are gated behind a future protocol version and cannot be used on-chain until Dijkstra.

On the Plutus V4 ScriptContext front, we are implementing a different encoding for product types using List instead of Constr, which makes them cheaper to decode. We are also improving the usability of the certifier in the uplc and plc tools.

Key Pull Requests Merged​

• Add Data.Constr casing to Case
• Add keepPolicies and dropPolicies builtins (CIP-0168)
• Cost model for policies (CIP-0168)
• Remove deferred builtin applications from the specification

Pull Requests In Progress​

• Builtin Pattern Matching (CIP-0194)
• Specify casing on built-in types
• Cost model for keepPolicies (CIP-0168)
• Cost model for dropPolicies (CIP-0168)
• Encode single constructor ledger types using Data.List
• Improve uplc/plc certifier usability
• Compile BuiltinData transparently in the plugin

High level summary​

• Leios prototype development (Treasury Funding Initiative 4: Ouroboros Leios Implementation):
  • Rewrote the logic that decides which Leios data to download from which peer. A node now bounds both the time it spends on each decision and the memory it holds for pending downloads. It also prioritises the newest Endorser Blocks, and randomises the order in which it asks for missing transactions, so that several requests do not saturate the same limits at once (#2237, for ouroboros-leios#1021).
  • Added a cache that holds the transactions of recent Endorser Blocks. A node that already saw a transaction does not need to download it again. The download logic reads from this cache and from the mempool (#2188, for ouroboros-leios#1021).
  • Fixed a second cause of unwanted disconnects between honest nodes. A syncing node has a chain tip that is too old to check young Leios messages. Such a node now stops asking for those messages until it catches up, instead of rejecting the replies (#2255, cardano-node#6673).
  • Capped the time the mempool spends to read a ledger state from disk. A block producer that waits on disk no longer delays the forging of its own block past its slot (#2217, for ouroboros-leios#911).
  • Extended the call tracing of the block forging and chain selection threads, and improved the dashboard that shows the results. This tells us where a prototype node spends its time on the Leios devnet (#2183, ouroboros-leios#1088, for ouroboros-leios#911).
  • Moved the classification and rendering of Leios trace messages from cardano-node into the consensus layer. We can now add or change a Leios trace message without a matching change in cardano-node (#2263).
  • Integrated the two changes above into the prototype node used on the devnet (cardano-node#6681).
• Storage layer (Treasury Funding Initiative 10: LSM including UTXO-HD):
  • Ledger state garbage collection now runs in its own thread. Before this change, the delay before a snapshot also blocked garbage collection, so a node kept every ledger state in memory until the snapshot completed (#2227).
  • Turned off IO wait accounting on the LSM storage backend. The accounting itself was a source of measurement error, see well-typed/blockio-uring#55 (#2247).
• Maintenance and support (Treasury Funding Initiative 17: Maintenance and Support):
  • Released ouroboros-consensus 4.2.0.0 (#2242).
  • Fixed a silent failure in the GetGenesisConfig query. Version 1.19.0.0 of cardano-ledger-shelley added a field to the Shelley genesis record without a change to the node-to-client protocol version. A client that linked the new ledger then waited forever for bytes that a node with the old ledger never sends. cardano-wallet hit this against node 11.0.1 and stopped following the chain. The query now sends the original 15 fields again, and the decoder accepts a reply from an already deployed node, so a client needs no node upgrade. The fix shipped in ouroboros-consensus 4.2.1.0 (#2251, #2257, #2258).

High level overview​

This week, the Mithril team completed the documentation of the recursive and non-recursive SNARK aggregation as well as the concatenation aggregation on the documentation website. They also completed the recovery of the ignored IVC prover input preparation tests and the split of the prove_all_scenarios monolith into separated tests. They continued adding the missing IVC off-circuit checks and the behavior tests for the IVC wiring, harmonizing the recursive and non-recursive SNARK implementations in STM, and started supporting an IVC follower aggregator joining the network at any time as well as the circuit verification key whitelist/revoke mechanism.

The team completed the new chain adapter that reads the on-chain configuration parameters, the adaptation of the legacy diffusion in the aggregator, the removal of these parameters from the aggregator configuration, the enhancements of the protocol configuration markers, and the infrastructure adaptation. They also completed the deterministic immutable compressed archives and the IPFS file uploader of the aggregator, continued implementing the IPFS file downloader of the client, fixed the deployment of the Cardano blocks and transactions signing configuration.

Finally, they kept upgrading to Cardano node 11.1 and updating the ed25519-dalek dependency.

Low level overview​

Features​

• Completed the issue Add documentation for recursive SNARK #3144
• Completed the issue Add documentation for non recursive SNARK #3155
• Completed the issue Add documentation for concatenation aggregation #3487
• Completed the issue Recover the ignored IVC prover input preparation tests #3466
• Completed the issue Replace prove_all_scenarios monolith with separated tests #3468
• Completed the issue Create new chain adapter to read on-chain configuration parameters #3393
• Completed the issue Adapt legacy diffusion in aggregator with on-chain configuration parameters #3394
• Completed the issue Remove the protocol configuration parameters in aggregator configuration #3395
• Completed the issue Enhancements protocol configuration markers #3503
• Completed the issue Create deterministic immutable compressed archives #3435
• Completed the issue Implement an IpfsFileUploader in the aggregator #3457
• Worked on the issue Add missing IVC off-circuit checks #3381
• Worked on the issue Add behavior tests for the IVC wiring #3465
• Worked on the issue Harmonize the recursive and non-recursive SNARK implementations in STM #3421
• Worked on the issue Support an IVC follower aggregator joining the network at any time #3446
• Worked on the issue Implement circuit verification key whitelist/revoke mechanism #3148
• Worked on the issue Implement an IpfsFileDownloader in the client #3459
• Worked on the issue Adapt documentation for on-chain configuration parameters #3398

Protocol maintenance​

• Completed the issue Adapt infrastructure for on-chain configuration parameters - Phase 1 #3397
• Completed on the issue Adapt infrastructure for on-chain configuration parameters - Phase 2 #3497
• Completed the issue Cardano blocks and transactions signing configuration is not deployed #3499
• Worked on the issue Upgrade to Cardano 11.1 #3346
• Worked on the issue Update ed25519-dalek dependency #3471