Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: better and DRYer matrixes #9235

Merged
merged 10 commits into from
Mar 25, 2024
Merged

ci: better and DRYer matrixes #9235

merged 10 commits into from
Mar 25, 2024

Conversation

neersighted
Copy link
Member

@neersighted neersighted commented Mar 24, 2024 •
edited
Loading

Hopefully drastically increase the maintainability of our CI workflows by DRYing up common bits, and taking better advantage of parallelism.

Note some big changes:

  • The main 'tests' workflow is now much wider and more parallel, and should surface meaningful results faster (e.g. mypy and pytest in parallel)
  • We now have a single 'Status' job intended to be used as a required check; this means we no longer need to keep updating branch protection rules
  • Likewise, skipped jobs should now actually show that they were skipped in the UI
  • pytest-xdist is now using 'logical' CPU cores as well (the distinction is less meaningful with VM vCPUs, so we should use all of them)
  • mypy data is cached, resulting in a 5x speedup on Windows
  • We're using the 'Poetry managed' venv for now, as this is all that setup-python can cache

@neersighted neersighted force-pushed the ci_revamp_3 branch 30 times, most recently from 34d55b5 to 25b12f4 Compare March 25, 2024 04:30
@neersighted neersighted force-pushed the ci_revamp_3 branch 3 times, most recently from 02d4252 to 1768ad1 Compare March 25, 2024 08:56
@neersighted neersighted requested review from abn, radoering, mkniewallner, Secrus and a team March 25, 2024 09:05
@neersighted neersighted force-pushed the ci_revamp_3 branch 5 times, most recently from 39f84e9 to 03158ad Compare March 25, 2024 09:57
@neersighted
tests: parallelize across logical cpu cores
9512e26
@neersighted
ci: DRY with composite actions
a076fb0
@neersighted
ci/tests: improve matrix and change filters
e4205b6
@neersighted
ci/tests: break individual test steps out into jobs
4348016
@neersighted
ci/tests: add test environment debug output
41ab53b
@neersighted
ci/tests: add workflow status job
7b0a316 
This job will wait until all other jobs in the workflow have finished,
and serves as the point of truth for 'required checks.'

This is a lot better than our current workflow, as it both accounts for
skipped jobs/matrixes (which are considered successful), and obviates
the need to change the branch configurations.
@neersighted
ci/tests: cache mypy
92534a5
@neersighted
ci/tests: DRY matrix with reusable workflow
c08e210
@neersighted
ci/tests: switch macOS to 13
87700fa
@neersighted
ci/cirrus: align with gha
858d803 
In particular, run mypy on FreeBSD in a parallel workflow to pytest.
@abn abn merged commit 7b69f14 into python-poetry:main Mar 25, 2024
60 checks passed
@neersighted neersighted deleted the ci_revamp_3 branch March 25, 2024 14:02
@github-actions GitHub Actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@abn abn abn approved these changes

@radoering radoering Awaiting requested review from radoering

@mkniewallner mkniewallner Awaiting requested review from mkniewallner

@Secrus Secrus Awaiting requested review from Secrus

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@neersighted @abn