Skip to content

feat: generate githubaction output before failing in case of breaking changes #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

alarvi
Copy link

@alarvi alarvi commented Apr 23, 2024

Motivation

As a user of the github breaking action, I was trying to use the example from the readme to create a workflow that would fail if a breaking change is introduced in a PR. Something like this:

- name: Running OpenAPI Spec diff action
  uses: oasdiff/oasdiff-action/breaking@main
  with:
    base: https://raw.githubusercontent.com/Tufin/oasdiff/main/data/openapi-test1.yaml
    revision: https://raw.githubusercontent.com/Tufin/oasdiff/main/data/openapi-test3.yaml

While this makes the workflow fail, it generates no output in the logs for the commit authors to review.

Fix

In this PR, the fail_on_diff flag is not passed to the oasdiff invocation to prevent the workflow from stopping too early. Instead, it waits until the end to check if the flag is true and any breaking change is detected.

@alarvi alarvi changed the title Generate githubaction output before failing in case of breaking changes feat: generate githubaction output before failing in case of breaking changes Apr 23, 2024
Comment on lines +89 to +91
if [ "$fail_on_diff" = "true" -a -n "$breaking_changes" ]; then
exit 1
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alarvi, thanks for the PR.
I agree that it's better to write the logs first in case of failure. However, the current implementation is handling logic that should be part of oasdiff. What if tomorrow it fails due to different logic? We would then need to implement that in both repositories. Additionally, what if oasdiff incorporates logic for response error codes?

Copy link
Author

@alarvi alarvi Apr 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@effoeffi Fair point. What would you suggest oasdiff does differently? I guess the ideal case for the action is that it never needs to call oasdiff twice, right?

Beyond that, what alternatives do we have to achieve both having the workflow fail and logs produced? I took inspiration from the tests and added this extra step to our workflow:

      - name: Check if there are any breaking changes
        run: |
          delimiter=$(cat /proc/sys/kernel/random/uuid | tr -d '-')
          output=$(cat <<-$delimiter
          ${{ steps.oasdiff.outputs.breaking }}
          $delimiter
          )
          if [ "$output" != "No breaking changes" ]; then
            echo "There are breaking changes in the OpenAPI spec"
            echo "$output"
            exit 1
          else
            echo "No breaking changes in the OpenAPI spec"
          fi

But now we have coupled the workflow with the output format of oasdiff, which isn't ideal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alarvi, as you mentioned, I see two options here:

  1. Calling oasdiff breaking twice: once to calculate breaking changes and write logs, then check if fail_on_diff is true and run it again.
  2. Running without fail_on_diff and using the breaking action output. In this case, it is coupled with the oasdiff action output message: "No breaking changes." Is that what you mean? If yes, you are correct, if we change this message, your code will break. We can add a comment to this message and a dedicated test that will fail in case this message changes.

Although it's not a perfect solution, I prefer the first one. What do you think?

Copy link
Contributor

@effoeffi effoeffi May 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alarvi could you please check this PR and let me know if it resolves your issue?

@effoeffi effoeffi self-assigned this Apr 24, 2024
@effoeffi effoeffi added the bug Something isn't working label Apr 24, 2024
@effoeffi effoeffi closed this May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants