Skip to content

CI Ledger RPC Tests #681

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Jul 25, 2022
Merged

CI Ledger RPC Tests #681

merged 42 commits into from
Jul 25, 2022

Conversation

ghzlatarev
Copy link
Contributor

@ghzlatarev ghzlatarev commented Jul 14, 2022

Description

closes: #644

  • Made the ledger-rpc-performance test a mocha test like the other ledger-rpc-correctness test
  • Added both tests to the CI, which expects the success return code 0
  • Both tests have a setup phase injecting items into storage and testing phase checking correctness or performance, In the performance test, both stages have a different timeout in order to make the test more robust.
  • Split the integration test runner in publish-draft-release workflow to 2 runners. One for calamari and one for dolphin. There are tests like the these ledger-rpc-tests that only apply to one of the runtimes.
  • Always runt he Dolphin integration test, but only run the Calamari inegration test on release tags or pushes to manta, in order to reduce unnecessary CI workload

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
  • Added one line describing your change in <branch>/CHANGELOG.md
  • Re-reviewed Files changed in the Github PR explorer.

Situational Notes:

  • If adding functonality, write unit tests!
  • If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
  • If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
  • If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.
  • If runtime changes, need to update the version numbers properly:
    • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
    • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
    • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
    • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
  • Verify benchmarks & weights have been updated for any modified runtime logics

Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
Signed-off-by: Georgi Zlatarev <[email protected]>
@ghzlatarev ghzlatarev self-assigned this Jul 14, 2022
@ghzlatarev ghzlatarev added A-ci Area: Continuous Integration A-testing Area: Testing-related Issues and PRs labels Jul 14, 2022
@ghzlatarev ghzlatarev marked this pull request as ready for review July 14, 2022 11:09
@ghzlatarev ghzlatarev requested a review from Apokalip July 14, 2022 11:48
@ghzlatarev
Copy link
Contributor Author

ghzlatarev commented Jul 21, 2022

@ghzlatarev can you elaborate why performance test need to be in CI?

Sanity check for regressions. The sync time can regress for a variety of reasons, including upstream code - storage, rpc, etc.
So it has an expected sync time now which is an average of 50 tries, https://github.com/Manta-Network/Manta/pull/681/files#diff-19d51960f59786f9c40532e38e3a902c64be0e21e3b84a46702bcb0718a20605R20
Usually it comes out to 520ms on the CI machines so the gave it a ~20% tolerance to 650

@ghzlatarev ghzlatarev requested a review from stechu July 21, 2022 06:02
Dengjianping
Dengjianping previously approved these changes Jul 22, 2022
@ghzlatarev ghzlatarev dismissed stale reviews from Dengjianping and Garandor via 1b13ba4 July 24, 2022 10:21
Signed-off-by: Georgi Zlatarev <[email protected]>
Apokalip
Apokalip previously approved these changes Jul 25, 2022
@ghzlatarev ghzlatarev requested a review from Garandor July 25, 2022 18:31
Copy link
Contributor

@Garandor Garandor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls calculate timeout as per comment

Signed-off-by: Georgi Zlatarev <[email protected]>
Garandor
Garandor previously approved these changes Jul 25, 2022
Signed-off-by: Georgi Zlatarev <[email protected]>
@ghzlatarev ghzlatarev merged commit 9e446bd into manta Jul 25, 2022
@ghzlatarev ghzlatarev deleted the ghzlatarev/ledger-rpc-ci-2 branch July 25, 2022 21:33
@Apokalip Apokalip added the L-changed Log: Issues and PRs related to changes label Jul 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ci Area: Continuous Integration A-testing Area: Testing-related Issues and PRs L-changed Log: Issues and PRs related to changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ledger RPC CI
5 participants