Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI improve nightly GHA workflow #761

Merged
merged 6 commits into from
Jan 4, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/actions/prepare-binaries/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Prepare polkadot and staking-miner-playground binaries

This action downloads the Polkadot and Staking miner binaries produced from workflows
`build-polkadot-for-nightly.yml` and `build-staking-miner-playground-for-nightly.yml` and puts them into the `$PATH`.
28 changes: 28 additions & 0 deletions .github/workflows/actions/prepare-binaries/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Prepare polkadot and staking-miner-playground binaries
description: Downloads and configures prebuilt Polkadot and Staking Miner Playground binaries
runs:
using: composite
steps:
- name: Download polkadot binaries
uses: dawidd6/action-download-artifact@e7466d1a7587ed14867642c2ca74b5bcc1e19a2d # v3.0.0
with:
workflow: build-polkadot-for-nightly.yml
name: nightly-polkadot-binary
path: ./tmp

- name: Download staking-miner-playground binary
uses: dawidd6/action-download-artifact@e7466d1a7587ed14867642c2ca74b5bcc1e19a2d # v3.0.0
with:
workflow: build-staking-miner-playground-for-nightly.yml
name: staking-miner-playground-binary
path: ./tmp

- name: Setup permissions and move to path
shell: bash
run: |
chmod u+x ./tmp/polkadot* ./tmp/staking-miner-playground
./tmp/polkadot --version
./tmp/staking-miner-playground --version
mkdir -p ~/.local/bin
mv ./tmp/polkadot* ~/.local/bin
mv ./tmp/staking-miner-playground ~/.local/bin
42 changes: 42 additions & 0 deletions .github/workflows/build-polkadot-for-nightly.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Build polkadot binary for nightly tests

on:
# Allow it to be manually ran to rebuild binary when needed:
workflow_dispatch: {}
# Run at 22pm every day for nightly builds.
schedule:
- cron: "0 22 * * *"

jobs:
tests:
name: Build polkadot binary
runs-on: ubuntu-latest
container: paritytech/ci-unified:bullseye-1.74.0-2023-11-01-v20231204
steps:
- name: checkout polkadot-sdk
uses: actions/checkout@v4
with:
repository: paritytech/polkadot-sdk

- name: Rust Cache
uses: Swatinem/rust-cache@v2
with:
workspaces: |
polkadot
cache-on-failure: true

- name: build polkadot binary
run: |
cargo build -p polkadot --release --features fast-runtime
./target/release/polkadot --version

- name: upload polkadot binary
uses: actions/upload-artifact@v3
with:
name: nightly-polkadot-binary
path: |
./target/release/polkadot
./target/release/polkadot-execute-worker
./target/release/polkadot-prepare-worker
retention-days: 2
if-no-files-found: error
44 changes: 44 additions & 0 deletions .github/workflows/build-staking-miner-playground-for-nightly.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Build staking miner playground binary for nightly tests

on:
# Run when changes to staking miner playground is made
push:
branches:
- main
paths:
- 'staking-miner-playground/**'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

# Allow it to be manually ran to rebuild binary when needed:
workflow_dispatch: {}
# Run at 22pm every week on Sunday's for nightly builds.
schedule:
- cron: "0 22 * * 6"

jobs:
tests:
name: Build staking-miner-playground
runs-on: ubuntu-latest
container: paritytech/ci-unified:bullseye-1.74.0-2023-11-01-v20231204
steps:
- name: checkout
uses: actions/checkout@v4

- name: Rust Cache
uses: Swatinem/rust-cache@v2
with:
workspaces: |
staking-miner-playground
cache-on-failure: true

- name: build staking-miner-playground binary
run: |
cd staking-miner-playground
cargo build --release --features test-trimming
./target/release/staking-miner-playground --version

- name: upload staking-miner-playground binary
uses: actions/upload-artifact@v3
with:
name: staking-miner-playground-binary
path: ./staking-miner-playground/target/release/staking-miner-playground
retention-days: 8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you could increase retention-rate to something like 30 days because the staking-miner playground hasn't been changed for a long time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Let it be 31, and binary will be rebuild on 1st day of every month :)

if-no-files-found: error
35 changes: 4 additions & 31 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Daily compatibility check against latest polkadot
name: Nightly test

on:
schedule:
Expand All @@ -8,21 +8,12 @@ on:
jobs:
nightly-test:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Free space on the runner
run: |
df -h
sudo apt -y autoremove --purge
sudo apt -y autoclean
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
df -h
- name: Get Polkadot and Staking Miner Playground binaries
uses: ./.github/workflows/actions/prepare-binaries

- name: Cache Rust dependencies
uses: swatinem/rust-cache@v2
Expand All @@ -32,26 +23,7 @@ jobs:

- name: Run nightly tests
run: |
sudo apt install -y protobuf-compiler
rustup target add wasm32-unknown-unknown
rustup component add rust-src
git clone -b master --depth 1 https://github.com/paritytech/polkadot-sdk.git polkadot-sdk
cd polkadot-sdk
cargo build -p polkadot --release --features fast-runtime
mkdir -p ~/.local/bin
sudo mv ./target/release/polkadot /usr/bin
sudo mv ./target/release/polkadot-execute-worker /usr/bin
sudo mv ./target/release/polkadot-prepare-worker /usr/bin
polkadot --version
cd -
rm -rf polkadot
cd staking-miner-playground
cargo build --release --features test-trimming
sudo mv ./target/release/staking-miner-playground /usr/bin
staking-miner-playground --version
cd -
RUST_LOG=info cargo test --workspace --all-features -- --nocapture
cargo clean

- name: Create an issue on failure
if: failure()
Expand All @@ -68,3 +40,4 @@ jobs:
# Look for new *open* issues in this search (we want to
# create a new one if we only find closed versions):
search_existing: open