Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JOSS Review] unit tests and output files #15

Open
bmcfee opened this issue Jun 5, 2020 · 0 comments
Open

[JOSS Review] unit tests and output files #15

bmcfee opened this issue Jun 5, 2020 · 0 comments

Comments

@bmcfee
Copy link

bmcfee commented Jun 5, 2020

reference review issue

In hitting the "automated tests" section, I noticed a couple of things about the unit test framework that could be improved.

  1. All tests seem to detect existence of the output file, but this does not confirm correct behavior. For many of the tests, it would be simple enough to detect, eg, changes in length or gain. For others (eg IR) you might need to do regression testing against known correct outputs.
  2. Minor thing, but your tests don't clean up after themselves. If you're using something like travis with build caching enabled, outputs from old runs could linger, and lead to spurious passing tests (because the files exist from a previous run). Since you're using pytest already, it wouldn't be too much work to use a tmp_path fixture to prevent this kind of behavior: https://docs.pytest.org/en/latest/tmpdir.html , or otherwise clean up old outputs after tests execute (pass or fail).

Point (2) would be much easier if the API was extended to allow the user to specify a target output path (#11). I see that you use the output filename to encode the deformation parameters, and letting a user specify the path exactly might make that difficult. I have some thoughts about how you might be able to accomplish both things (ie via string interpolation), but that might be out of scope for the testing issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant