Add OSV code scanner and make minor adjustments to CI (#992)

* Run all nightly tests from a common top-level workflow

This removes the individual cron schedule entries from the workflows
that run at night, and instead invokes them from `nightly.yaml`.

* Rename workflows to use .yaml extension consistently

`.yaml` is actually the official file extension for YAML.

* Add OSV scanner workflow

* Adjust names and trigger conditions

* Tell yamllint the long lines are okay

* Rework organization some more

* Set up another problem matcher

* Add .editorconfig file (#990)

This adds settings based on some common-sense values and the
OpenFermion project's current conventions, such as line length and use
of spaces instead of tabs.

* Add shellcheck for shell script linting (#991)

* Add .shellcheckrc

This adds a config file for .shellcheckrc, and sets a few optional
flags to try to make shellcheck slightly more thorough.

* Add shellcheck problem matcher

* Add a heading to help break up the text a little

* Add shellcheck to CI checks

* Edit CONTRIBUTING.md (#993)

This adds a warning about the CLA implications for GenAI code, and
tries to add some more info about the contribution process.

* Add markdownlint config file (#995)

This adds a `.markdownlintrc` configuration file for
[markdownlint](https://github.com/igorshubovych/markdownlint-cli), a
popular linter for Markdown files. The settings here try to be as
close to the Google style as possible while still being compatible
with GitHub-flavored Markdown.

This same markdownlint file is used in Cirq and other Quantumlib
projects.

* Fix warnings reported by shellcheck (#997)

There are no functional changes in this commit; all changes are to
address lint reported by shellcheck.

* CI checks need to be run if ci.yml changes (#998)

The tests for changed files needs to consider changes to the workflow
definitions too, because a change to the workflow definition could
involve changing options or other aspects of the CI tests that might
change the check results.

* Add problem matchers for Mypy, Pytest, and Black (#1000)

* Add problem matchers for mypy and pytest

* Add problem matcher for Black

* Pass input sha to format-incremental

If the user supplied a sha in a manual workflow run, we need to pass
that to `format-incremental` or else it won't compare files to the
correct revision.

* Prefer single quotes in YAML

Due to how GitHub Actions syntax works, I'm finding it safer to use
single quotes for YAML values and (usually) double quotes in shell
scripts.

* Tighten format of "run:" for one-liners

It seems slightly reasier to read the code whenn single-line shell
commands are put on the same line as the `run:`, instead of being put
on a separate line.

* More setup, editing, configuration, renaming, tweaking

.github/workflows/ci.yml → .github/workflows/ci.yaml

@@ -36,6 +36,9 @@ on:
         description: Max compat version of Python
         type: string
 
+# Declare default permissions as read only.
+permissions: read-all
+
 concurrency:
   # Cancel any previously-started but still active runs on the same branch.
   cancel-in-progress: true
@@ -426,6 +429,9 @@ jobs:
       - name: Install requirements
         run: pip install -r dev_tools/requirements/envs/pytest.env.txt
 
+      - name: Set up Pytest output problem matcher
+        run: echo '::add-matcher::.github/problem-matchers/pytest.json'
+
       - name: Run code coverage tests
         run: check/pytest-and-incremental-coverage
 

.github/workflows/codeql.yaml

@@ -1,11 +1,17 @@
 # Summary: configuration for CodeQL.
+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-name: Nightly CodeQL code scan
-run-name: Perform nightly CodeQL code scan
+name: CodeQL code scan
+run-name: Do ${{inputs.reason}} CodeQL code scan
 
 on:
-  schedule:
-    - cron: '37 13 * * 5'
+  # Allow calling from nightly.yaml.
+  workflow_call:
+    inputs:
+      # Why is this workflow being called?
+      reason:
+        type: string
+        required: false
 
   # Allow manual invocation.
   workflow_dispatch:
@@ -20,7 +26,7 @@ concurrency:
 
 jobs:
   codeql:
-    name: Run CodeQL analysis
+    name: Run
     runs-on: ubuntu-24.04
     permissions:
       security-events: write
@@ -37,4 +43,4 @@ jobs:
       - name: Perform CodeQL Analysis
         uses: github/codeql-action/analyze@b56ba49b26e50535fa1e7f7db0f4f7b4bf65d80d # v3
         with:
-          category: "/language:Python"
+          category: '/language:Python'

.github/workflows/nightly-pytest.yml → .github/workflows/nightly-pytest.yaml

@@ -1,16 +1,27 @@
-# Summary: reusable workflow called by nightly.yml
-#
+# Summary: reusable workflow called by nightly.yaml.
 # This workflow expects input values passed by nightly.yml.
 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-name: Nightly full tests – Pytest subworkflow
+name: Nightly full tests – Pytest matrix
+run-name: Run ${{inputs.reason}} Pytest full tests
 
 on:
   workflow_call:
     inputs:
+      # CLI argument string to pass to pytest.
       args:
         type: string
-        required: true
+        required: false
+      # Why is this workflow being called?
+      reason:
+        type: string
+        required: false
+
+  # Allow manual invocation.
+  workflow_dispatch:
+
+# Declare default permissions as read only.
+permissions: read-all
 
 jobs:
   Pytest:
@@ -24,8 +35,8 @@ jobs:
         # These Python versions were carefully selected by trial and error
         # to be available on as many os/arch combos as possible, while also
         # making all necessary Python dependencies available on those combos.
-        # TODO: add "3.13.1" once Cirq 1.5 is released.
-        python-version: ["3.10.11", "3.11.9", "3.12.7", "3.13.1"]
+        # TODO: add '3.13.1' once Cirq 1.5 is released.
+        python-version: ['3.10.11', '3.11.9', '3.12.7']
         os: [ubuntu-22.04, ubuntu-22.04-arm, macos-14, windows-2022]
         arch: [x64, arm64]
         exclude:
@@ -48,6 +59,7 @@ jobs:
           - os: ubuntu-22.04-arm
             arch: arm64
             python-version: 3.12.7
+
     steps:
       - name: Check out a copy of the OpenFermion git repository
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
@@ -64,9 +76,9 @@ jobs:
       - name: Install OpenFermion Python requirements
         run: |
           pip install -r dev_tools/requirements/envs/pytest.env.txt
-          echo "::group::List of installed pip packages and their versions"
+          echo '::group::List of installed pip packages and their versions'
           pip list
-          echo "::endgroup::"
+          echo '::endgroup::'
 
       - name: Install cirq-core (current stable version)
         run: |

.github/workflows/nightly.yaml

@@ -0,0 +1,55 @@
+# Summary: run nightly tests and scans.
+#
+# This workflow runs nightly to run tests & scans on the OpenFermion codebase.
+# It can also be invoked manually via the "Run workflow" button at
+# https://github.com/quantumlib/OpenFermion/actions/workflows/nightly.yaml
+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+name: Nightly full tests
+run-name: Run nightly tests and code scans in ${{github.repository}}
+
+on:
+  schedule:
+    - cron: '15 2 * * *'
+
+  # Allow manual invocation.
+  workflow_dispatch:
+
+# Declare default permissions as read only.
+permissions: read-all
+
+jobs:
+  cirq-stable:
+    name: Pytest Cirq stable release
+    uses: ./.github/workflows/nightly-pytest.yaml
+    with:
+      args: ''
+      reason: nightly
+
+  cirq-pre:
+    name: Pytest Cirq pre-release
+    uses: ./.github/workflows/nightly-pytest.yaml
+    with:
+      args: '--pre'
+      reason: nightly
+
+  codeql:
+    name: CodeQL code scan
+    uses: ./.github/workflows/codeql.yaml
+    permissions: write-all
+    with:
+      reason: nightly
+
+  scorecard:
+    name: Scorecard code scan
+    uses: ./.github/workflows/scorecard.yaml
+    permissions: write-all
+    with:
+      reason: nightly
+
+  osv:
+    name: OSV code scan
+    uses: ./.github/workflows/osv-scanner.yaml
+    permissions: write-all
+    with:
+      reason: nightly

.github/workflows/osv-scanner.yaml

@@ -0,0 +1,71 @@
+# Summary: run Open Source Vulnerabilities (OSV) code scan.
+#
+# For more examples and options, including how to ignore specific
+# vulnerabilities, see https://google.github.io/osv-scanner/github-action/.
+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+name: OSV code scan
+run-name: Do ${{inputs.reason}} OSV code scan
+
+on:
+  merge_group:
+    types:
+      - checks_requested
+
+  # Allow calling from nightly.yaml.
+  workflow_call:
+    inputs:
+      # Why is this workflow being called?
+      reason:
+        type: string
+        required: false
+
+  # Allow manual invocation.
+  workflow_dispatch:
+
+# Declare default permissions as read only.
+permissions: read-all
+
+jobs:
+  osv-scan:
+    name: Run
+    runs-on: ubuntu-22.04
+    timeout-minutes: 15
+    permissions:
+      # Needed to upload the results to code-scanning dashboard.
+      security-events: write
+      # Read commit contents
+      contents: read
+      actions: read
+    steps:
+      - name: Check out a copy of the git repository
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          persist-credentials: false
+
+      - name: Run OSV analysis
+        # yamllint disable rule:line-length
+        uses: google/osv-scanner-action/osv-scanner-action@f8115f2f28022984d4e8070d2f0f85abcf6f3458 # v1.9.2
+        continue-on-error: true
+        with:
+          scan-args: |-
+            --format=json
+            --output=osv-results.json
+            --recursive
+            --skip-git
+            ./
+
+      - name: Run osv-scanner-reporter
+        # yamllint disable rule:line-length
+        uses: google/osv-scanner-action/osv-reporter-action@f8115f2f28022984d4e8070d2f0f85abcf6f3458 # v1.9.2
+        with:
+          scan-args: |-
+            --output=osv-results.sarif
+            --new=osv-results.json
+            --gh-annotations=true
+            --fail-on-vuln=true
+
+      - name: Upload to code-scanning dashboard
+        uses: github/codeql-action/upload-sarif@dd746615b3b9d728a6a37ca2045b68ca76d4841a # v3.28.8
+        with:
+          sarif_file: osv-results.sarif

.github/workflows/ossf-scorecard.yaml → .github/workflows/scorecard.yaml

@@ -1,28 +1,36 @@
 # Summary: workflow for OSSF Scorecard (https://github.com/ossf/scorecard).
-# Scorecard's use is suggested in Google's GitHub guidance (go/github-docs).
+#
+# Scorecard checks for security best practices. Its use is suggested in
+# Google's GitHub guidance.
+# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-name: Scorecard supply-chain security
-run-name: Analyze code for Scorecard
+name: Scorecard analysis
+run-name: Do ${{inputs.reason}} Scorecard analysis
 
 on:
-  schedule:
-    - cron: '19 20 * * 6'
+  merge_group:
+    types:
+      - checks_requested
+
+  # Allow calling from nightly.yaml.
+  workflow_call:
+    inputs:
+      # Why is this workflow being called?
+      reason:
+        type: string
+        required: false
 
   # Allow manual invocation.
   workflow_dispatch:
 
 # Declare default permissions as read only.
 permissions: read-all
 
-# Cancel any previously-started but still active runs on the same branch.
-concurrency:
-  cancel-in-progress: true
-  group: ${{github.workflow}}-${{github.event.pull_request.number||github.ref}}
-
 jobs:
   scorecard:
-    name: Perform Scorecard analysis
+    name: Run
     runs-on: ubuntu-22.04
+    timeout-minutes: 15
     permissions:
       # Needed to upload the results to code-scanning dashboard.
       security-events: write