{Codecov upload issue debugging} - Ensuring the Codecov UI is able to react to data from a commit that is re-run

[rohan-at-sentry]

Problem

The Codecov UI is essentially static after the CI finishes for a commit - which is problematic in the following scenarios

• when the user re-runs a commit on CI and therefore re-uploads coverage (typically done to eliminate flaky tests or fix a temporary issue with a coverage report

We want to ensure the Codecov UI updates data for the commit in this scenario.

Why is it important?

Not seeing consistency on what you see on the PR reflect what you see confuses people.

Path forward

• ensure the coverage report errors on the commits tab are shown/not shown based on if the latest coverage report upload (new run, same commit) was successful or not

  • Related issue (Implement the Upload processing status UI component #2522) that adds coverage report errors to the UI on the commits tab
  • Errors shown should be from the latest upload to error out

• Update top level coverage statistics for the commit and PR with result from latest result

• Set notify_error to true for all new customers (the default appears to be false). This allows any issues with uploads to surface on the PR comment as well --> serves as a warning signal to users that coverage reporting on the UI may be impacted.

[rohan-at-sentry]

Bring in platform if we need to store things to a db (for perf reasons)

[Adal3n3]

Meeting with @suejung-sentry 12/20:

• Design turn: maybe we could add a Re-run button to the UI, allowing Codecov to compute the latest processed reports and display them. Currently, Codecov cannot detect the latest re-run from GitHub, especially when the user re-runs a single specific step. With this approach, @suejung-sentry would need to check if it’s feasible.
• ToDO: @Adal3n3 open an issue: Investigate if it’s feasible to add Re-run button for updating the latest coverage report in commit page #3142
  Example: link