Great Lakes Data Assimilation

[shorvath-noaa]

This PR adds a new data assimilation method specific to the Great Lakes. The Great Lakes are not currently routed in t-route due to complexities with their size. Previously, the level pool module was used for them, but due to some parameters being intentionally set to 0, they produce 0 outflow. This new method uses the best available observations (the lakes don't have their own outflow gages, so it uses gages that are nearby). Lakes Superior and Michigan/Huron use nearby USGS gages, lake Erie uses a Canadian gage, and Lake Ontario uses forecasted flows from the International Lake Ontario-St. Lawrence River Board.

Because the level pool module does not work on these lakes, we default to using climatological values in the event that there are no good observations. We've implemented an 11 day persistence method similar to USGS/USACE reservoirs, meaning the DA module will allow an observation to persist for up to 11 days, after which it will default to climatology if no new observations are available.

Additions

reservoir_GL_da.py

• Script containing the DA function used for the Great Lakes. This was initially based on the persistence DA method used for USGS/USACE reservoirs.

Removals

Changes

mc_reach.pyx

• Added a new reservoir type (6) for the Great Lakes.

DataAssimilation.py

• Added a new DA class, great_lakes_DA. This handles the preprocessing of observations and updating observation and parameter data frames after loops in __main__.py.

HYFeaturesNetwork.py

• Added methods for preprocessing the Great Lakes in the connections dictionary.
• Load climatology data from a file.
• Find crosswalk information between Great Lakes and gages.
• NOTE: Some of this was hard-coded into t-route due to lack of all necessary information in the hydrofabric. New release of the hydrofabric should contain the necessary information, so this will need to be updated...

nhd_io.py

• Added a new function, get_GL_obs_from_timeslices(). Almost identical to the existing get_obs_from_timeslices(), except it doesn't do any interpolation.

rfc_lake_gage_crosswalk.py

• Added a new function, get_great_lakes_climatology(), that loads preprocessed climatology values for the Great Lakes.

compute.py & main.py

• Pass great lakes data frames through to mc_reach.pyx.

Testing

1. Created a test case on VPU4 ("nextgen_04.gpkg", hydrofabric v2.1). 18 day run with DA enabled to verify that all aspects of the DA module are working properly. Below show two examples, Lake Superior (lake ID: 4800002, observations roughly every 15 minutes) and Lake Erie (lake ID: 4800006, observations once per day). Red dots are observations, the solid line is the t-route output, and the dotted line is the climatology value. The DA modules checks for a new output every hour, so some observations are not assimilated. But the output line matches the observations every hour until no good observations are found. The last good value is persisted for 11 days, then the output matches climatology. Once a new, good observation is found, data assimilation resumes.

Screenshots

Notes

Todos

Checklist

• PR has an informative and human-readable title
• Changes are limited to a single goal (no scope creep)
• Code can be automatically merged (no conflicts)
• Code follows project standards (link if applicable)
• Passes all existing automated tests
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• Visually tested in supported browsers and devices (see checklist below 👇)
• Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
• Reviewers requested with the Reviewers tool ➡️

Testing checklist

Target Environment support

• Windows
• Linux
• Browser

Accessibility

• Keyboard friendly
• Screen reader friendly

Other

• Is useable without CSS
• Is useable without JS
• Flexible from small to large screens
• No linting errors or warnings
• JavaScript tests are passing

[AminTorabi-NOAA]

Seems like there is a conflict with current branch.

[shorvath-noaa]

Seems like there is a conflict with current branch.

Resolved.

[AminTorabi-NOAA]

This function is very similar to get_obs_from_timeslices function. Is it possible to merge these two?

[shorvath-noaa]

Yeah, it's actually identical just without the interpolation step. We could merge them, but I do want to avoid interpolating for the Great Lakes, so we'd have to alter the original function to only interpolate under certain conditions. I chose this method for now for simplicity, but I'm open to re-working this function.