-
-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Verbose Unit Testing Update #723
Conversation
This PR touches files which potentially affect the outcome of the tests of an exercise. This will cause all students' solutions to affected exercises to be re-tested. If this PR does not affect the result of the test (or, for example, adds an edge case that is not worth rerunning all tests for), please add the following to the merge-commit message which will stops student's tests from re-running. Please copy-paste to avoid typos.
For more information, refer to the documentation. If you are unsure whether to add the message or not, please ping |
Due to test failures, I also modified |
@ErikSchierboom I've had a look at the test runner and I think the fix to remove the top-level category from the test name on the website is fairly straightforward. I had standardized the top-level testset description for all the exercises in this PR as |
I think I'm fine with the standardized description, as the context is indeed known. Before we merge this, could you add a test case with the new runtests.jl structure to https://github.com/exercism/julia-test-runner/tree/main/test/fixtures? The test runner should work well with both the old and the new format. |
I've tried to add a test case to the test-runner and to fix the test name issue. I opened a PR(closed automatically by github-actions) that has both of these fixes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the test runner PR!
Cheers! |
That's just the setup workflow, so likely just an intermittent error. |
It was not, but #733 fixes it. |
I've updated all the the
runtests.jl
files to use verbose unit testing similar to what is mentioned in this issue. However, there are two slight modifications:runtests.jl
files in all of the following folders:exercises/concept
,exercises/concept.wip
,exercises/practice
, instead of just those inexercises/practice
"test"
in order to limit the impact on what is printed in the test name on the website, while still remaining informative for those who use the CLI.The original description for the top level testset was the exercise name. However, this can be quite long (taking up a lot of space in the website test results), and is also rather irrelevant since the student presumably already knows what exercise they are doing.
tests
is shorter, standardized and informative on the CLI since it corresponds to a summary of all test outcomes (as shown below).