Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

run frame-omni-bencher overhead command in CI for all runtimes in the runtime matrix #7459

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

clangenb
Copy link
Contributor

@clangenb clangenb commented Feb 5, 2025

As defined in #6347, we want to run the omni-benchers overhead command in the CI to be sure that it keeps working.

Closes #6347

Todo: The broken tests will be fixed after merging:

@clangenb clangenb requested review from a team as code owners February 5, 2025 08:18
@clangenb clangenb force-pushed the cl/run-frame-omni-bencher-in-ci branch 2 times, most recently from 6d9dde6 to dc7194d Compare February 5, 2025 09:01
@clangenb clangenb force-pushed the cl/run-frame-omni-bencher-in-ci branch from dc7194d to 9000c0c Compare February 5, 2025 09:03
@paritytech-review-bot paritytech-review-bot bot requested a review from a team February 5, 2025 09:42
@clangenb
Copy link
Contributor Author

clangenb commented Feb 5, 2025

Some tests failed because not all chains have development genesis presets. I shall add them in the same PR.

@mordamax
Copy link
Contributor

mordamax commented Feb 5, 2025

Some tests failed because not all chains have development genesis presets. I shall add them in the same PR.

may be this flag needs to be added?
https://github.com/paritytech/polkadot-sdk/blob/master/.github/workflows/runtimes-matrix.json#L108

if so you can reuse bench_flags from the matrix

@clangenb
Copy link
Contributor Author

clangenb commented Feb 5, 2025

Thanks a lot for the hint! I thought/hoped so too initially, however the flags are different, unfortunately.

I can overwrite the development preset with another one, but for the overhead command "none" is unfortunately not an option. It does need a preset.

@clangenb
Copy link
Contributor Author

clangenb commented Feb 5, 2025

I noticed that there is an open issue regarding the missing genesis presets. I extracted a separate PR for that:

@clangenb clangenb marked this pull request as draft February 5, 2025 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run omni-bencher overhead cmd in CI
2 participants