Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verbose Unit Testing Update #723

Merged
merged 4 commits into from
Apr 30, 2024
Merged

Verbose Unit Testing Update #723

merged 4 commits into from
Apr 30, 2024

Conversation

depial
Copy link
Contributor

@depial depial commented Apr 16, 2024

I've updated all the the runtests.jl files to use verbose unit testing similar to what is mentioned in this issue. However, there are two slight modifications:

  1. I've changed the runtests.jl files in all of the following folders: exercises/concept, exercises/concept.wip, exercises/practice, instead of just those in exercises/practice
  2. I have made the description for the top level testset "test" in order to limit the impact on what is printed in the test name on the website, while still remaining informative for those who use the CLI.

The original description for the top level testset was the exercise name. However, this can be quite long (taking up a lot of space in the website test results), and is also rather irrelevant since the student presumably already knows what exercise they are doing. tests is shorter, standardized and informative on the CLI since it corresponds to a summary of all test outcomes (as shown below).

image

Copy link
Contributor

This PR touches files which potentially affect the outcome of the tests of an exercise. This will cause all students' solutions to affected exercises to be re-tested.

If this PR does not affect the result of the test (or, for example, adds an edge case that is not worth rerunning all tests for), please add the following to the merge-commit message which will stops student's tests from re-running. Please copy-paste to avoid typos.

[no important files changed]

For more information, refer to the documentation. If you are unsure whether to add the message or not, please ping @exercism/maintainers-admin in a comment. Thank you!

@depial
Copy link
Contributor Author

depial commented Apr 16, 2024

Due to test failures, I also modified concept.wip/encounters/runtests.jl by moving the struct definitions outside the local scope of the testset wrapper. All tests are passing now.

@depial depial mentioned this pull request Apr 17, 2024
10 tasks
@depial
Copy link
Contributor Author

depial commented Apr 24, 2024

@ErikSchierboom I've had a look at the test runner and I think the fix to remove the top-level category from the test name on the website is fairly straightforward. I had standardized the top-level testset description for all the exercises in this PR as tests in order to minimize any effects of this not being fixed, but I could change the descriptions if desired. Let me know if there is a preference.

@ErikSchierboom
Copy link
Member

@ErikSchierboom I've had a look at the test runner and I think the fix to #680 (comment) from the test name on the website is fairly straightforward. I had standardized the top-level testset description for all the exercises in this PR as tests in order to minimize any effects of this not being fixed, but I could change the descriptions if desired. Let me know if there is a preference.

I think I'm fine with the standardized description, as the context is indeed known. Before we merge this, could you add a test case with the new runtests.jl structure to https://github.com/exercism/julia-test-runner/tree/main/test/fixtures? The test runner should work well with both the old and the new format.

@depial
Copy link
Contributor Author

depial commented Apr 26, 2024

I've tried to add a test case to the test-runner and to fix the test name issue. I opened a PR(closed automatically by github-actions) that has both of these fixes.

Copy link
Member

@ErikSchierboom ErikSchierboom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the test runner PR!

@ErikSchierboom ErikSchierboom merged commit a9e391b into exercism:main Apr 30, 2024
11 checks passed
@depial
Copy link
Contributor Author

depial commented Apr 30, 2024

Thanks for the test runner PR!

Cheers!

@depial
Copy link
Contributor Author

depial commented Apr 30, 2024

I don't know why, since all the checks passed here earlier, but CI is now failing for Julia 1.6 - macOS-latest

image

@depial depial deleted the verbose branch April 30, 2024 22:52
@ErikSchierboom
Copy link
Member

That's just the setup workflow, so likely just an intermittent error.

@ErikSchierboom
Copy link
Member

It was not, but #733 fixes it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants