Adapt pysteps to allow for postprocessing plugins

[joeycasey87]

Added an infrastructure.py file to the postprocessing folder which should operate in the same way as the interface file does for the importers in the io folder. The postprocessor file is currently effectively empty as no postprocessor plugins have been added yet. A line has been added to the init file so that it should act similar to the init file in the io folder.

[dnerini]

Suggested change

	from postprocessors import *
	from .postprocessors import *

[dnerini]

just wondering if "postprocessors" is a good name? sounds a bit too vague, particularly as a submodule of "postprocessing" ... can we try to be a bit more specific? perhaps use "diagnostic" instead? what do you think? @ladc ?

[ladc]

Yes good point - "diagnostic" makes sense if we also want to add other postprocessors in the future which are not purely diagnostic (such as bias correction methods).

[codecov]

Codecov Report

Attention: Patch coverage is 85.71429% with 16 lines in your changes missing coverage. Please review.

Project coverage is 84.18%. Comparing base (e332585) to head (3586f56).

Files with missing lines	Patch %	Lines
pysteps/postprocessing/interface.py	78.87%	15 Missing ⚠️
pysteps/tests/test_interfaces.py	96.55%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #405      +/-   ##
==========================================
+ Coverage   84.17%   84.18%   +0.01%     
==========================================
  Files         160      162       +2     
  Lines       13058    13170     +112     
==========================================
+ Hits        10991    11087      +96     
- Misses       2067     2083      +16     

Flag	Coverage Δ
unit_tests	84.18% <85.71%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[joeycasey87]

Hi @dnerini, could you please review this pull request when you have the time? The extended version of the cookiecutter plugin template which can be used to create importer, diagnostic postprocessing, and nowcast plugins is nearly finished but I require the postprocessing interface to be merged to ensure that everything is behaving correctly when tested. Cheers

[ladc]

Thanks for this contribution! We discussed how to make it more generic and @joeycasey87 will try to refactor it tomorrow to allow for various kinds of post-processors including new ensemblestats and bias correction methods, for example. To be continued.

[dnerini]

what's the status of this PR? should we mark it as a draft?

[ladc]

Apologies for the force-push, but I have done a rebase to the latest master version. The plan is now to clean up some of the dummy code (maybe move this into test, but most likely remove it entirely), add the necessary test(s) in test_plugins_support.py and hopefully merge it into the master branch soonish, thanks in advance to @FelixE91 #nopressure

For the tests, we're now only checking the importer cookiecutter template. The new precipitation type plugin was created using the diagnostics template (I suppose), so we should add a test for this new template ( see also https://github.com/pySTEPS/pysteps/blob/e3325854dae006b1e78eb43a6d2203f2b0b71560/pysteps/tests/test_plugins_support.py#L17C1-L17C20 )

Question: do we really need a separate template for all the plugin types? (Diagnostics, ensemblestats ... which are all in fact postprocessors)? Could we have one for any kind of postprocessors? Or even better, one cookiecutter template for all kinds of plugins which could even expose multiple functions (e.g. diagnostics and visualisation)? Not urgent at all.