124 posts tagged with "hydra"

What did the team achieve?​

• Released 2.3.0, headlined by the reapplyTx snapshot-processing speedup, node configuration via YAML file, native PaymentExtendedKey (HD wallet) key support #2583, and the batch of rotation, Blockfrost and network-layer resilience fixes from the last two updates. Note that all the subsequently listed work has not yet been released
• Merged the performance stack #2776-#2780, in five reviewable pieces:
  • accumulator commitments are now computed through the rust-accumulator FFI (bit-identical, pinned by equivalence and golden tests) and updated incrementally from the previous confirmed snapshot, taking the 4k-UTxO commitment from 2.13s to 25ms #2780
  • BREAKING: broadcast messages are batched into a single etcd value, one Raft commit per batch over a reused gRPC connection. This bumps the network protocol version to 2, so all members of a head must upgrade together #2778
  • maxTxsPerSnapshot raised from 100 to 1000, leader-side only so no coordinated upgrade is needed #2777
  • SQLite event encoding moved onto the write-behind thread, keeping serialisation of large SnapshotRequested events off the node loop between processing a ReqSn and broadcasting the AckSn #2779
  • plus the measurement and benchmark infrastructure the whole stack was driven from #2776
• Implemented selective partial fanout: a new PartialFanout client input lets you distribute a chosen subset of the closed head's UTxO, with a HeadPartiallyFannedOut server output telling clients whether the node keeps draining or awaits the next selection, a FanningOut head status, and a matching TUI flow. Only the node that issued the command drives the step, so observers can't steamroll UTxO another party deliberately left behind #2750, #2333, with a docs page to go with it #2794
• BREAKING: --deposit-period is now embedded in the on-chain OpenDatum at Init time, so all nodes in a head must agree on it; a mismatch emits IgnoredHeadInitializing and the node ignores the head, the same way contestation period mismatches already worked #2734
• Added --deposit-activation to decouple activation from the deadline calculation: it controls only the Inactive -> Active transition while --deposit-period keeps its deadline and expiry role, giving three independent windows (maturity, active, recovery). The default of 3600s reproduces the old behaviour exactly #2802, #2744
• POST /commit now rejects deposits that could never be claimed. A dry-run increment is checked against the L1 maximum transaction and serialized value sizes, returning a new DepositTooLarge error. Previously the deposit would land on-chain and the increment claiming it could never be submitted, wedging further incremental commits and locking the funds until recovery after the deadline #2807
• Fixed long-running nodes rejecting every layer 2 Plutus transaction carrying a validity bound: the era history queried at startup has a forecast horizon (36 hours on mainnet), and once past it those transactions failed with PastHorizon until restart. The last era is now unbounded for L2 ledger time conversions #2803, #2798
• BREAKING (WebSocket API): removed the SyncedStatusReport server output, which the node pushed on every block. Sync status is now edge-triggered via NodeUnsynced/NodeSynced and Greetings.chainSyncedStatus, and the continuous drift value became a hydra_chain_drift_seconds metric alongside a new hydra_chain_last_block_timestamp_seconds so a stalled chain backend is detectable even while drift is frozen #2769, #2749
• Made hydra-chain-observer version-aware: it detects which protocol version an observed transaction belongs to by matching head and deposit script hashes against a registry in networks.json covering 0.13.0 through 2.2.0, removing the compile-time coupling to a single version and the hydra-plutus dependency altogether #2740
• Fixed pasting into hydra-tui text fields (bracketed paste mode, so a paste arrives as one event and typing c into an address no longer cancels the dialog) and stopped it submitting stale form values when the entered address or amount was invalid #2770
• Made the mac release binaries runnable without nix: they were hard-linking libraries out of /nix/store and simply wouldn't launch on a plain machine #2795, #2461
• Bumped Mithril to 2630.0 and switched to the v2 database backend, since the aggregator no longer certifies v1 #2808
• Docs fixes: the transaction-cost benchmark page is now generated by the docs build instead of being dropped in as raw markdown afterwards, so it renders with the current theme and stops listing removed transactions #2799, and the fanout cost table now sweeps party counts #2796

What's next?​

• Complete the typst + Agda specification work, split into a reviewable stack: a protocol fix forbidding a commit and a decommit in flight simultaneously #2784, the prose migration from LaTeX to Typst #2785, the Agda formalisation itself #2786, a hydra-agda package extracted via MAlonzo with a CI gate #2787, and differential tests checking the node and validator against that reference #2788
• Switch the event store to CBOR #2767 and offer an opt-in binary encoding on the client API #2762
• Iterate on ADR 34, covering the separation of hydra-chain-observer into a service shared by several nodes #2771
• Make rejectLowDeposits a single pass rather than quadratic #2805
• Get a release out with the performance stack and selective partial fanout

High-level Summary​

The user-friendly hydra-node configuration file has been merged, letting operators run a head from a single YAML file and diff configs against peers at a glance. The rest of the period was mostly hardening: deposit and recovery chain observations are now scoped to the head that produced them (closing a long-standing cross-head contamination issue), the Blockfrost backend retries on HTTP errors instead of crashing and no longer chokes on some inline datums, and the event store's rotation logic got two fixes - a race between VACUUM and streaming client history, and a deadlock in the network layer's outbound message queue that could silently halt broadcasting.

What did the team achieve?​

• Merged the user-friendly hydra-node configuration file: nodes can now be configured via YAML, making it easy to spot differences between peers' setups #2581, #2296
• Scoped deposit and recover chain observations to the current head: Open/Closed states only record OnDepositTx events for their own headId and Idle silently drops the rest, while recovery of a deposit from a previous head keeps working in every state, including mid-fanout and while a new head is running #2743, #2606
• Made the Blockfrost chain backend more resilient: HTTP errors are now retried instead of crashing the node #2729, #2477, and a follow-up fixed inline datum decoding that could throw on some UTxOs #2754, #2751
• Fixed a race between the event log's VACUUM-based rotation and the API server streaming client history over the same SQLite connection, which could take a node down on reconnect #2755
• Fixed a deadlock in the network layer's PersistentQueue: a false Eq result in popPersistentQueue could leave the broadcast queue stuck at capacity with no log output; it now pops unconditionally by index and emits PersistentQueueFull / PersistentQueueLoadFailed for visibility, and a failed file delete no longer crashes the network component #2742
• Event log rotation now archives the outgoing database to a numbered old-state/hydra-<logId>.db file instead of deleting it, restoring the old file-based persistence's pre-rotation backup behaviour #2748, #2747
• Added a fuel verification key option to hydra-tui so the funds tab can show both the head's ada and a separate fuel UTxO #2731
• Moved the spec back into the hydra repo, dropping the separate flake input #2733

What's next?​

• Merge some in-progress performance work: computing accumulator commitments via the rust-accumulator FFI instead of off-chain PlutusTx, batching etcd broadcasts onto shared connections, and raising the per-snapshot transaction cap - early results show sustained throughput roughly doubling on a 1k-UTxO head #2752
• Make hydra-chain-observer version-aware by detecting script hashes, so a single process can watch heads from any historical Hydra version instead of running one per version #2740
• Enforce --deposit-period consistency across head nodes by embedding it in the on-chain OpenDatum #2734
• Accept PaymentExtendedKey (HD wallet keys) #2583
• Work on the updated version of the spec (typst+Agda)

High-level Summary​

We've published the 2.2.0 release! It bundles up the partial fanout work and the hydra-tui revamp from the last couple of updates, together with a much richer benchmark suite (real-world TPS metrics, a Mixed UTxO generator, and a new matrix sub-command that sweeps cluster sizes and UTxO shapes, published to a scenario benchmarks page). On top of that we unified the on-chain membership checks around the BLS accumulator, which shrinks the snapshot-signing tuple and removes a pile of now redundant hashing, and started optimising snapshot processing so script-heavy heads don't pay to re-evaluate Plutus they've already validated.

What did the team achieve?​

• Released 2.2.0, packaging partial fanout, the hydra-tui revamp, and the new benchmarking work, with the changelog and network config updated accordingly
• Unified the membership checks: the three separate UTxO hash fields (utxoHash, alphaUTxOHash, omegaUTxOHash) collapse into a single accumulatorHash derived from the BLS accumulator, shrinking the snapshot signing tuple from 7 fields to 4, and full fanout now verifies outputs via a KZG membership proof (the same path as partial fanout) rather than a hash comparison #2619
• Started a snapshot-side throughput optimisation: transactions already validated on receipt are now re-applied via reapplyTx, skipping redundant Plutus script evaluation on the hot path, with a fallback to full application when a commit or decommit changes the active UTxO #2717
• Extended PR benchmarking: a "TPS" count (both waiting for TxValid and firing as fast as possible then counting confirmed-snapshot txns) #2635, and a benchmark comparison comment showing the PR-vs-master difference #2721
• Added a head-state visualisation tool for inspecting single states and diagnosing multi-party scenarios #2719
• Added a mustNotMintOrBurn guard to the Increment and Decrement validator transitions, consistent with all the other head transitions #2718, #2697
• Further validator fixes: CloseInitial now constrains the accumulator commitment, and the HeadStatus schema and Haskell types agree on Final #2712
• Removed leftover CollectCom/Abort handling missed in the original "directly open head" work #2709
• Switched to makeIsDataIndexed instead of unstableMakeIsData for stable on-chain constructor indices #2724, #2713
• TUI fixes: the increment dialog no longer re-queries L1 UTxO state every time (it reuses the cached value and lets you refresh if you want) #2726, plus more small fixes #2710 and a fix for flakey TUI tests #2708
• Merged a handful of simple fixes across smaller issues #2715
• Added deposit and recover transition tests and tidied up some others #2711
• CI and tooling: more nix caching by building derivations with -Werror to match nix flake check #2720, added etcd to the exes shell so smoke tests can run a system etcd #2722, and skipped the preprod-with-mithril path while it's offline #2723

What's next?​

• Add a test that probes how large a UTxO set we can actually (partially) fan out #2616
• Continue the still-open node-observing-unrelated-Heads deposit issue #2606
• Finish the user-friendly hydra-node configuration file #2581
• Accept PaymentExtendedKey (HD wallet keys) #2583

High-level Summary​

Big news is that we have merged the partial fanout work!. After many months of design and review, heads with large UTxO sets can now be fanned out in multiple steps, each verified on-chain via a BLS accumulator membership proof, with the chunk size determined dynamically by a binary search so each step packs in as many outputs as will fit. This removes the long-standing limit on UTxOs per head. Alongside that, the hydra-tui got a significant refactor, the test suite moved to tasty, and a handful of correctness fixes landed around cross-head event contamination and recovered deposits leaking back into L2.

What did the team achieve?​

• Merged partial fanout, including the on-chain validators, BLS accumulator membership proofs, and the PartialFanoutTx / FinalPartialFanoutTx transactions #2324
• Made the fanout chunk size fully dynamic via binary search over fitting tx size, replacing the hardcoded threshold #2617
• Significantly revised hydra-tui: pending-deposit recovery from Open and Closed/Final, dark/light theme toggle persisted to disk, event-history filter, tab navigation, and a per-tab rendering refactor #2590
• Switched the test suite from hspec to tasty #2620
• Validated headId in the aggregate function to prevent replayed events from a previous head corrupting the state of a newly opened one #2618
• Fixed a bug where a recovered incremental-commit deposit could reappear in the L2 UTxO after sideloading a snapshot, making the same UTxO spendable on both L1 and L2 #2630, #2629
• Made etcd writes more resilient to resolve a pumba-style network-partition test failure #2627
• Bumped rust-accumulator to fix musl build issues #2622
• Disabled nix store optimisation on CI to work around a macOS dyld issue #2626
• Sped up repeated linting and normal building, and added a small CLAUDE.md #2625
• Updated docs and changelog to reflect the partial fanout implementation, and documented the remaining large-UTxO limitation #2631

What's next?​

• Get a new release out!
• Add a test that probes how large a UTxO set we can actually (partially) fan out #2616
• Continue the still-open node-observing-unrelated-Heads deposit issue #2606
• Land the user-friendly hydra-node configuration file #2581
• Accept PaymentExtendedKey (HD wallet keys) #2583

High-level Summary​

This week the team shipped hydra-node 2.1.0 - a significant release that brings improved deposit security, the new SQLite-backed event store, a ~7% reduction in snapshot confirmation latency, and compatibility with cardano-node protocol version 12+. We encourage all operators to upgrade. Alongside the release, work continues on two long-standing items: partial fanout and a user-friendly configuration file for hydra-node to replace the current sprawl of command-line flags.

What did the team achieve?​

• Released hydra-node 2.1.0, which includes improved deposit security, the SQLite-backed event store, a ~7% reduction in snapshot confirmation latency, and compatibility with cardano-node protocol version 12+ release 2.1.0
• Upgraded to cardano-api 11.1 / cardano-node 11.0.1 to support protocol version 12+ #2607
• Switched from a lazy to a strict Map to reduce memory pressure during transaction processing #2599

What's next?​

• Continue work on partial fanout — choosing the UTxO set to fan out and final code review #2593, #2324, #2573
• Reduce disk usage in hydra-node #2601, #2600
• Investigate a node observing/persisting deposit events from unrelated Heads #2606
• User-friendly hydra-node configuration via config file #2581
• Accept PaymentExtendedKey (HD wallet keys) #2583