86 posts tagged with "ledger"

High level summary​

• CIP-118 - Nested Transactions:

  Over the past two weeks the Ledger team has continued to push the Dijkstra era forward on multiple fronts. The CDDL specification of block was updated to match CIP-0176 - Non-segregated Block Body Serialization, a dedicated block_body was added and an EncCBORGroup instance for DijkstraBlockBody was added to unblock downstream integration in Consensus. Serialization for DijkstraTx was fixed up as well. We also followed up on the Plutus context performance work introduced in the previous update by integrating StAnnTx (the state annotated transaction representation that memoizes the Plutus parts of validation) into the LEDGER, UTXOW, UTXO and MEMPOOL rules across every era. With this integration the performance benefits of the earlier memoization work are now realized end to end. Further work in this area will let us rip the benefits of this optimization upon block construction as well, not just during initial validation. On top of that, several guards were added to the Plutus V1-V3 context translation so that any of the new Dijkstra features that those older language versions cannot represent (subtransactions, direct deposits, balance intervals, etc.) cause translation to fail explicitly rather than silently producing a wrong context. We also did preparation work for running conformance tests in Dijkstra and started reorganizing the test hierarchy around a new class-based ImpSpec approach that enforces a common set of tests across all eras.

• LedgerHD

  The streaming infrastructure for injecting large amounts of data from genesis files, which was started in the previous cycle, has been completed. It now covers stake pools and the remaining state needed for the testing and benchmarking work that targets reading data in constant memory from disk and paves the way for further direct injection into on-disk ledger state.

• CIP-165 - Canonical Ledger State:

  The expansion of the Canonical Ledger State continued with two more namespaces being added under the new namespace-based query interface, entities/dreps/v0 and entities/stake_pools/vrf_key_hashes/v0.

• Maintenance

  We've expanded examples and added golden tests for all of the migrated queries from Consensus to Ledger done last cycle. Furthermore we did the same for transaction examples. All of this effort not only will let us remove golden tests unrelated to the Consensus component from the ouroboros-consensus repository, but also improve their quality.

  In addition to features we published the changelog for the recent cardano-node-11.0 release, made ZeroTreasuryWithdrawals a permanent governance check for every era starting with Conway, instead of not having it in the bootstrap phase. We switched ChainData and Attributes to use ByteArray instead of ByteString to avoid pinned-memory fragmentation. A few NoThunks instances were tightened up where consensus needs them for storing on disk and relaxed where the values are already forced to normal form. Finally, we fixed a cuddle failure on Plutus Constr data and a bug in its definition in the CDDL spec discovered by our new generator-based tooling.

Low level summary​

Features​

• PR-5787 - Remove redundant import
• PR-5785 - Relax header protocol version for testnets
• PR-5786 - Changelog for cardano-node-11.0
• PR-5733 - Dijkstra era CDDL for block, add block_body
• PR-5789 - Integrate StAnnTx in rules across all eras
• PR-5777 - Full support of data injection from genesis files using streaming
• PR-5793 - Change DijkstraTx decoder to support variable length encoding
• PR-5669 - Fail PlutusV1-V3 translation for non-empty subTransactionsTxBody
• PR-5797 - Make ZeroTreasuryWithdrawals a permanent check in gov state transation rule
• PR-5700 - Add entities/dreps/v0 namespace
• PR-5808 - Relax NoThunks instance for BlockTransitionError
• PR-5781 - Fail Plutus V1-V3 translation with guards
• PR-5813 - Implement EncCBORGroup instance for DijkstraBlockBody
• PR-5812 - Switch ChainData and Attributes to use ByteArray
• PR-5811 - Fix TranslateErainstance for DijkstraEra CertState
• PR-5743 - Add entities/stake_pools/vrf_key_hashes/v0 namespace
• PR-5823 - Add NoThunks instances for ChainTransitionError and ChainPredicateFailure
• PR-5816 - Import Rules as qualified
• PR-5799 - Add example transaction golden test for each era

Testing​

• PR-5778 - Add golden tests for queryConstitution
• PR-5761 - Expand upon transaction examples by adding missing fields.
• PR-5794 - Add golden tests for the rest of the query result types
• PR-5801 - Fix failing cuddle test for Plutus data Constr.
• PR-5775 - Preparation for conformance testing in Dijkstra
• PR-5803 - Start to re-organize test hierarchy: ImpSpec
• PR-5826 - Fix trace construction in TestChain

Infrastructure and releasing​

• PR-5790 - Bump slackapi/slack-github-action from 3.0.2 to 3.0.3 in the actions group
• PR-5796 - Disable doctest to unblock master
• PR-5806 - Bump cuddle to 1.7.0.0
• PR-5817 - Bump urllib3 from 2.6.3 to 2.7.0 in /doc
• PR-5822 - Remove weird hyperlinked space between badges
• PR-5758 - Refactor testing in GitHub CI

High level summary​

Ledger team has made significant progress on Ledger rules for Nested Transactions as well as translation of newer features to Plutus context. We've also introduced a major change to how Plutus context is prepared for each transaction entering a mempool, which when fully integrated will result in significant performance improvements. On the efforts of expanding Ledger we have transferred all of the ledger state queries from Consensus component to the Ledger component, as well as introduced an initial stage of streaming injection functionality that we'll be needed for testing and benchmarking, especially for on disk storage where huge amount of data needs to be injected into ledger state without any impact on the operating memory.

Aside from features we have fixed a few more cddl bugs that we discovered with our new AntiGen and cuddle tools. Improved examples for golden tests and golden tests facilities.

Low level summary​

Features​

• PR-5692 - Adjust Dijkstra UTXOW rule to subtransactions
• PR-5684 - Remove unused FromCBOR/ToCBOR instances
• PR-5572 - Add streaming interface to era transition for initial funds injection
• PR-5737 - Fix treasury donations handling for SubTransactions
• PR-5755 - Clean up CddlSpec, fix Datum and AccountBalanceIntervals generators
• PR-5736 - Fix DecCBOR instances to reject ProtVer values exceeding the era maximum
• PR-5636 - Memoize Plutus related parts of transaction validation
• PR-5764 - Bring all remaining queries from consensus over to ledger
• PR-5767 - [Leios Prototype] Backwards compat fix
• PR-5757 - Remove allowLeftOver flag from binaryGetDecoder and simplify IP address decoders
• PR-5754 - Fail PlutusV1-V3 translation for direct deposits and balance intervals
• PR-5770 - Simplify Dijkstra constraints
• PR-5751 - UTXO with subtransactions
• PR-5747 - Add gov/proposals/roots/v0 namespace
• PR-5742 - Add entities/stake_pools/v0 namespace

Testing​

• PR-5749 - Fix and re-enable BbodysSec
• PR-5687 - Convert pool related CHAINExamples to ImpTests
• PR-5732 - Expand transaction content in examples in Test.Cardano.Ledger.[era].Examples
• PR-5753 - Use hspec-golden for Alonzo golden tests
• PR-5760 - Golden testing facilities for JSON and CBOR
• PR-5731 - Add negative tests for Metadatum int decoding range
• PR-5765 - Add a custom generator to cost_models, create Huddle.Gen module

Infrastructure and releasing​

• PR-5750 - Run tests for ghc 9.6.7 only
• PR-5756 - Bump actions/upload-pages-artifact from 4 to 5 in the actions group
• PR-5766 - Drop x86_64-darwin from Hydra builds
• PR-5768 - Bump slackapi/slack-github-action from 3.0.1 to 3.0.2 in the actions group
• PR-5594 - Test failure summaries in CI
• PR-5774 - Handle single failing suite case in CI test failure summaries

High level summary​

These were our main areas of focus last period on the Ledger project:

Maintenance​

We worked really hard in order to prepare Ledger for the cardano-node-10.7.0 release that had all of the bug fixes and features needed for the upcoming van Rossem Hard Fork into protocol version 11. Furthermore, we prepared Ledger for a subsequent cardano-node-10.7.1 release that had some of the performance issues resolved, which were identified in the Ledger component of 10.7.0 release.

We've also identified and fixed a few bugs, most of which were in CDDL specification. This was largely thanks to the recently integrated tool AntiGen, that we developed specifically for creating negative property tests for Ledger serialization.

Besides bug fixes, we've added quite a few nice optimizations, which do result in noticeable speed up in chain validation and replay.

Some very nice improvements were implemented for the Ledger API used by Consensus, which will result in clearer and safer boundaries between those two components.

CIP-118 - Nested Transactions:​

Two more major Ledger rules UTXO and UTXOW were defined for processing transactions in the Dijkstra era that now take sub-transactions into consideration.

CIP-165 - Canonical Ledger State:​

Fixes were applied to canonical ledger state representation for the Constitutional Committee

Low level summary​

Features​

• PR-5654 - Normalize StakeRefPtr addresses better and improve UTxO decoding
• PR-5647 - Change B constructor of Metadatum from ByteString to ByteArray
• PR-5655 - Add ToJSON/FromJSON instances for NonEmptyMap and NonEmptySet
• PR-5630 - Remove distinct_bytes hack, make CDDL reflect reality in previous eras
• PR-5560 - *EraBlockHeader typeclasses deprecating BHeaderView
• PR-5658 - Optimize Text byte length check in metadata and other places
• PR-5632 - Forecast API
• PR-5671 - Remove NoThunks instances for predicate failures and ContextError types
• PR-5635 - ScriptPurpose aware memoization of TxInfo
• PR-5676 - Remove state update from UTXOS
• PR-5641 - Avoid unnecessary computation of PlutusWithContext
• PR-5672 - Make ContextError constructors from Alonzo to Dijkstra era lazy
• PR-5614 - SUBUTXO implementation
• PR-5668 - SUBUTXOW fixes
• PR-5704 - Backport: revert ConwayAccountState type unrolling
• PR-5697 - Remove 5 redundant NativeScript era ~ Timelock era constraints (#5319)
• PR-5665 - Fix metadatum decoder and chain_code CDDL
• PR-5681 - Add/refactor committee-related canonical namespaces
• PR-5725 - Revert decoders to a more efficient implementation
• PR-5716 - Move applyTick into its own ApplyTick typeclass
• PR-5719 - Add protocol version validation to createInitialState
• PR-5729 - Fix ConwayAccountState overhead
• PR-5727 - Add storable instances

Testing​

• PR-5656 - Translate all transaction building examples in our test files to use Lens interface.
• PR-5609 - Adequate conformance to Version restriction
• PR-5667 - Fix Proposal generation to generate HardForkInitiation with bounded major version
• PR-5695 - Update formal-ledger-specifications and re-enable tests
• PR-5529 - Add simpleRewards reproducer from cardano-db-sync and migrate TwoPools example to an ImpTest
• PR-5715 - Bump cuddle to 1.5.0.0

Infrastructure and releasing​

• PR-5653 - Add cabal-gild and shellcheck as part of the pre-commit hook in our Nix devShell.
• PR-5648 - Changelog for cardano-node-10.7
• PR-5657 - Migrate scripts to cardano-ledger-release-tool
• PR-5599 - Enable ghc 9.14 to be used for tests in CI and locally
• PR-5677 - Bump requests from 2.32.4 to 2.33.0 in /doc
• PR-5675 - Add nixfmt in our Github CI checks
• PR-5670 - Add CHANGELOG and cabal file updater skill
• PR-5691 - Bump pygments from 2.19.1 to 2.20.0 in /doc
• PR-5702 - Update contribution policy to note how contributions should be vetted
• PR-5703 - Add a CI workflow for checking nix hashes
• PR-5708 - Bump the actions group with 3 updates
• PR-5709 - Update slack-github-action configuration for v2 onward
• PR-5712 - Fix missed version bump for allegra package
• PR-5713 - Stop running bump-changelogs on release branches
• PR-5666 - Refactor transaction and transaction body of examples in Test.CardanoLedger.<era>.Examples.hs.
• PR-5717 - claude: improve changelog update skill
• PR-5710 - Update microlens dependency
• PR-5726 - Update ghc-9.12 on CI and Nix to ghc-9.12.4, and cabal-3.16 to cabal-3.16.1.0
• PR-5728 - Backport of fixes for cardano-node-10.7.1
• PR-5741 - Fix nix build failure in plutus-preprocessor
• PR-5721 - Leios: Make the (leios enhanced) block decoding backwards compatible

High level summary​

This period Ledger has progressed on Nested Transaction by implementing one more Ledger rule. We've also integrated our new testing library AntiGen into Ledger, which allows us to preform negative property testing on our decoders against CDDL specification, consequently improving confidence in the quality of our binary decoders now and for future eras. We've also progressed on Canonical Ledger State by adding one more namespace. Major milestone was releasing of all Ledger packages to CHaP for the upcoming cardano-node-10.7 release, which has all the features in it for the upcoming intra-era hard fork to protocol version 11.

Low level summary​

Features​

• PR-5532 - Remove default implementation for the DecCBOR class
• PR-5612 - Introduce gov/proposals/v0 namespace
• PR-5611 - Use IPv4 and IPv6 from cardano-base instead of iproute
• PR-5608 - SUBUTXOW implementation
• PR-5627 - Remove redundant UTXO predicate failures
• PR-5629 - Remove default implementation of fromPlutusData in ToPlutusData typeclass
• PR-5645 - Add PlutusV4 to AlonzoScript decoder

Testing​

• PR-5568 - Add AntiGen tests to Conway CDDL
• PR-5628 - Implement custom ToExpr instance for Mismatch datatype.
• PR-5620 - Update formal-ledger-spec and enable Utxos conformance tests

Infrastructure and releasing​

• PR-5615 - Bump markdown from 3.7 to 3.8.1 in /doc
• PR-5624 - Bump cardano-ledger-alonzo-test package version
• PR-5622 - Post-release CHANGELOG updates

High level summary​

Through some extra testing we've identified and fixed a few benign bugs. We've also identified and implemented another restructure in the ledger state that improves performance and will further simplify LedegrHD implementation. Besides that we've also made further progress on these initiatives:

• CIP-118 - Nested Transactions:

  Two major Ledger rules were defined for processing sub-transactions.

• CIP-165 - Canonical Ledger State:

  Two more namesspaces for canonical ledger state were implemented.

Low level summary​

Features​

• PR-5575 - Introduce gov/committee/v0 canonical namespace
• PR-5584 - Use AccountId for stake pools
• PR-5566 - Tighten PParam lenses
• PR-5591 - Extras for working with maps
• PR-5595 - Version restriction
• PR-5596 - Remove SUBUTXOS rule
• PR-5579 - Support gov/canonical/pparams/v0 namespace
• PR-5589 - add notes on Praos slot leader election for historic reasons
• PR-5585 - Subsume delegations into active-stake
• PR-5544 - SUBLEDGER implementation

Testing​

• PR-5587 - Set scriptIntegrityHash to 0 to match the behavior of Agda in conformance tests
• PR-5590 - Fix nightly nothunks tests for snapshot

Infrastructure and releasing​

• PR-5292 - Support ghc 9.14
• PR-5607 - Fix nix builds
• PR-5600 - Fix gen-hie caching