config: add new DEFAULT_CONFIG_PARAMETERS_SCRIPT_URL

[tlvu]

Require Ouranosinc/PAVICS-e2e-workflow-tests#142

[fmigneault]

Waiting on Ouranosinc/PAVICS-e2e-workflow-tests#142 deliberation.

Will have to check with @perronld also about the impact.

If we need to reboot the compose each time a change is applied to the script to take effect, this will not be a viable solution. On the other hand, if it doesn't need a reboot (ie: it applies dynamically), why is there even a need to supply it here? It couldn't just be a variable in the existing scripts that toggle/filter based on a "blacklist" of notebooks?

[tlvu]

If we need to reboot the compose each time a change is applied to the script to take effect, this will not be a viable solution.

No need to reboot Jenkins each time. This is dynamic on each build request.

On the other hand, if it doesn't need a reboot (ie: it applies dynamically), why is there even a need to supply it here? It couldn't just be a variable in the existing scripts that toggle/filter based on a "blacklist" of notebooks?

Good question ! I did not implement this just for blacklisting notebooks. I implemented a generic pluggable default override that can do anything (blacklist, adding new, performing some pre/post processing, ... ).

The intend is a generic solution so we do not have add more commits each time we need yet a new feature and spawn again a new variable to toogle and that would work for any repos, even those unknown at this point by the current Jenkins config.

Each change is trying to approach us to resolving this issue Ouranosinc/PAVICS-e2e-workflow-tests#57

[fmigneault]

I'm somewhat worried about the feature in the long term.

For now, we already have a way to override some variables "in-line" to an execution with Jenkins parameters, and a way to define a different branch for PAVICS-e2e-workflow-tests. This should provide enough leverage to test alternative behaviors.

If we have situations where more code is needed to "undo" certain operations to handle edge cases, I'd argue that the problem lies in the main PAVICS-e2e-workflow-tests scripts in the first place. Hacking them via another default config, which will probably not suit our needs at some point, so it would need another test-override config, just means we'll have many places where code doing some test preparation is defined. Those "multi-layered patches" will never be merged back into the PAVICS-e2e-workflow-tests scripts, and everything will just be harder to maintain or understand between us, because nothing runs the same way anywhere.

[tlvu]

For now, we already have a way to override some variables "in-line" to an execution with Jenkins parameters, and a way to define a different branch for PAVICS-e2e-workflow-tests. This should provide enough leverage to test alternative behaviors.

I see the confusion ! You can do it because your pipeline have access to PAVICS' env.local so you override those Jenkins params programmatically in env.local.

But in practice, outside of your pipeline, there is no way for a regular user to perform this programmatic override because a standalone Jenkins do not have access to the target env.local. That feature is a way for regular user to perform programmatic override !

Those "multi-layered patches" will never be merged back into the PAVICS-e2e-workflow-tests scripts, and everything will just be harder to maintain or understand between us, because nothing runs the same way anywhere.

Well isn't this the same with our birdhouse-deploy? Any component is like a "multi-layered patches" because any subsequent component can override anything (variables, docker-compose directives) in all previous components. We also have a bunch of custom private components in external repos that we also will never merge back to birdhouse-deploy because it's specific to us !

Our PAVICS deployment is slightly different in term of components enabled but once a component is enabled, they have the same config, except for the Jupyter image list of course, we have a lot more. I think the "nothing runs the same way anywhere" is how we ensure each org can customize PAVICS to their liking. Same for this Jenkins I hope.