Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Playwright: Use new GH Pages actions #1504

Merged
merged 23 commits into from
Feb 4, 2025
Merged

Playwright: Use new GH Pages actions #1504

merged 23 commits into from
Feb 4, 2025

Conversation

sunker
Copy link
Contributor

@sunker sunker commented Jan 31, 2025

What this PR does / why we need it:

This PR uses the new actions to publish Playwright reports to Github Pages rather than uploading them as artifacts or deploying them to GCS.

Playwright tests running as part of plugin-e2e CI (playwright.yml)
Reports where at least one of the tests are now being deployed to GH Pages. A report comment is added to the PR with a summary of all the runs (see example at the bottom of this PR.

Scaffolded plugins
Like today, we don't want to upload reports as artifacts or deploy them to GH Pages automatically, as reports may include secrets (no directly after them being scaffolded, but if actively changed by plugin authors). For that reason, the necessary actions (upload-report-artifacts and deploy-report-pages) are added to the CI.yml workflow, however the upload-report is set to false meaning the report won't be uploaded to GH artifacts. While running tests as part of CI, a comment with a test result matrix will be added including a note describing how to enable publishing to GH pages.

Screenshot 2025-02-03 at 09 16 38

Documentation

The example workflows showcased in the e2e CI docs are updated. There's also a detailed step by step instructions on how to add support for publishing reports. This will be useful for all plugins that have been scaffolded with an older version of create-plugin.

Example of this feature being used in external plugins:
gh-pages

Which issue(s) this PR fixes:

Fixes #1456

Special notes for your reviewer:

Since a while back, I've had a PR open to run e2e tests that run as part of create-plugin changes in two Grafana versions. I've updated this PR too so that it also publishes reports to GH actions.

📦 Published PR as canary version: Canary Versions

✨ Test out this PR locally via:

npm install website@3.6.0-canary.1504.c3ff863.0
npm install @grafana/create-plugin@5.15.0-canary.1504.c3ff863.0
# or 
yarn add website@3.6.0-canary.1504.c3ff863.0
yarn add @grafana/create-plugin@5.15.0-canary.1504.c3ff863.0

@sunker sunker added the no-changelog Don't include in changelog and version calculations label Jan 31, 2025
Copy link

github-actions bot commented Jan 31, 2025

Hello! 👋 This repository uses Auto for releasing packages using PR labels.

✨ This PR can be merged and will trigger a new minor release.
NOTE: When merging a PR with the release label please avoid merging another PR. For further information see here.

.github/workflows/playwright.yml Fixed Show fixed Hide fixed
.github/workflows/playwright.yml Fixed Show fixed Hide fixed
@sunker sunker added minor Increment the minor version when merged release Create a release when this pr is merged labels Jan 31, 2025
.github/workflows/playwright.yml Fixed Show fixed Hide fixed
- uses: actions/checkout@v4
- name: Publish report
## todo: change to main branch once the pr is merged
uses: grafana/plugin-actions/playwright-gh-pages/deploy-report-pages@main

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'E2E tests' step
Uses Step
uses 'grafana/plugin-actions/playwright-gh-pages/deploy-report-pages' with ref 'main', not a pinned commit hash
@grafana grafana deleted a comment from github-actions bot Jan 31, 2025
@sunker sunker removed the no-changelog Don't include in changelog and version calculations label Jan 31, 2025
uses: grafana/plugin-actions/playwright-gh-pages/upload-report-artifacts@main
if: $\{{ always() && !cancelled() }}
with:
upload-report: false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means the report won't be uploaded to GH artifacts, and therefore not deployed to GH pages in the next job.

if: ${{ github.repository_owner == 'grafana' && (failure() && steps.run-tests.outcome == 'failure') }}
uses: grafana/plugin-actions/publish-report@main
- name: Upload e2e test summary
uses: grafana/plugin-actions/playwright-gh-pages/upload-report-artifacts@main

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'E2E tests' step
Uses Step
uses 'grafana/plugin-actions/playwright-gh-pages/upload-report-artifacts' with ref 'main', not a pinned commit hash
@sunker sunker marked this pull request as ready for review February 3, 2025 13:55
@sunker sunker requested a review from a team as a code owner February 3, 2025 13:55
…arted.md

Co-authored-by: Timur Olzhabayev <timur.olzhabayev@grafana.com>
Copy link

github-actions bot commented Feb 4, 2025

Playwright test results

Image Name Version Result Report
grafana-dev 11.6.0-221763
grafana-enterprise 11.5.1
grafana-enterprise 11.2.6
grafana-enterprise 10.1.10
grafana-enterprise 9.2.20
grafana-enterprise 8.5.27
Troubleshooting

404 when clicking on View report

By default, the deploy-report-pages Action deploys reports to the gh-pages branch. However, you need to take an extra step to ensure that GitHub Pages can build and serve the site from this branch. To do so:

  1. Go to the Settings tab of your repository.
  2. In the left-hand sidebar, click on Pages.
  3. Under Source, select Deploy from a branch, then choose the gh-pages branch.

This action needs to be completed manually in order for your GitHub Pages site to be built and accessible from the gh-pages branch. Once configured, GitHub will automatically build and serve the site whenever new reports are deployed.

@sunker sunker merged commit ae7756e into main Feb 4, 2025
34 checks passed
@sunker sunker deleted the use-gh-pages-actions branch February 4, 2025 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
minor Increment the minor version when merged release Create a release when this pr is merged
Projects
Status: 🚀 Shipped
Development

Successfully merging this pull request may close these issues.

Feat: Add docs and tooling for publishing Playwright reports to GH pages
2 participants