Skip to main content

5 posts tagged with "system-test"

View All Tags

· 2 min read
Martin Kourim

High level summary​

Since the last update (2023-03-26) we did more improvements on our Test Framework, added more automated tests, kept analyzing and fixing nightly jobs failures, tested the 1.35.7 and 8.0.0 releases.

Workstreams​

Full list of closed PRs

Framework improvements:​

  • Make cardano-submit-api REST API service available and run corresponding regression tests by default
  • Update cabal build testing for 8.0.0
  • Add support for testing governance SPO poll
  • Use Poetry for Python dependency management
  • Check status of known GH issues during tests runtime and finish the test accordingly
  • Generate topology files with both IP addresses and DNS names
  • Log issues like failure to start a cluster instance and report errors during tests runtime

Testing improvements​

  • Add rollback testing:
    1. global consensus is reached after rollback, in situation where less than securityParam blocks were produced since cluster split
    2. global consensus is not reached when more than securityParam blocks were produced, and the result is permanent fork
  • Add tests to governance SPO poll commands

Node:​

DB-Sync:​

  • improvements in db-sync sync tests
  • improvements in db-sync functional tests

· One min read
Martin Kourim

High level summary​

Since the last update (2023-1-13) we did more improvements on our Test Framework, added more automated tests, kept analyzing and fixing nightly jobs failures, tested the 1.35.5 and 1.35.6 releases (and their predecessing release candidates) and ran tests on branch with the UTxO-HD functionality.

Workstreams​

Framework improvements:​

  • improvements in searching for unexpected error messages in log files during test run
  • added support for UTxO-HD testing
  • added support for the new P2P topology format
  • added local cluster variant that has the same properties as mainnet (epoch length, fees, etc.)
  • added preliminary support for the Conway era
  • added support for running tests on the Preview testnet

Full list of closed PRs

Node:​

  • ran tests on node branch with the UTxO-HD functionality
  • tested the 1.35.5 release
  • tested the 1.35.6 release

DB-Sync:​

  • improvements in db-sync sync tests
  • improvements in db-sync functional tests

· One min read
Dorin Solomon

High level summary​

During the last 2 weeks we did more improvements on our Test Framework, ran some sanity tests related to the P2P Single Relay functionality.

We also update the Node & DB-Sync sync tets to build with Nix as the prebuilt files are no longer available at PR level.

Workstreams​

Framework improvements:​

  • extended the cardano-node-tests with the ability for anybody to fork the repo and run all our System Tests on GitHub Actions
  • added 2 new nightly pipelines - nightly-mixed and nightly-p2p - details here
  • some optimizations on how our regression tests are scheduled on pytest workers and how cluster instances are assigned to the tests;

=== 743 passed, 67 skipped, 24 xfailed in 9166.64s (2:32:46) === to === 753 passed, 67 skipped, 14 xfailed in 4654.80s (1:17:34) ===

Node:​

  • ran a couple of sanity runs of CLI a& sync tests on a local branch with P2P Single Relay enabled
  • started the preparations for testing the next tag - details here

DB-Sync:​

  • some improvements on db-sync sync tests

· One min read
Dorin Solomon

High level summary​

During the last 2 weeks we did more improvements on our Test Framework, planned the testing of the P2P Single Relay functionality, and also tested some DB-Sync tags.

Workstreams​

Framework improvements:​

  • moved the System Test CLI Pipelines from BuildKite to Github Actions
  • improved the reporting tools to support the rerun of the failled tests and update of the reports
  • added support for Github API in report-aggregator, so reports will be generated from the GitHub nightly jobs from now on
  • added support for mixed topology - P2P, legacy, mixed topologies
  • planned the P2P Single Relay system test activities
  • added support to start regression tests with PV8 + better selection of tests

DB-Sync:​

  • confirmed that DB-Sync release 13.0.5 is compatible with Node release 1.35.4 + Protocol Version 8 (on the Preview environment)
  • tested a couple db-sync tags - 13.1.0.0-rc1, 13.1.0.0-rc2

· One min read
Dorin Solomon

High level summary​

We have been focused on:

  • Fully opening our test results (on top of the existing tests & tools):
    See cardano-node-tests webpage.
  • Started to test and automate the new functionalities added in the 1.35.4-rc1 node tag
    See test results tracking page.
  • Made some improvements to the automated db-sync sync tests
    See db-sync tests.
  • Multiple cleanups and updates to the cardano-node-tests framework
  • Updated the nightly pipelines for the cardano-node-tests after the Babbage HF