Meeting Minutes from February 22, 2024

[dezidizon]

Agenda:

1. Branch security under the CentralValleyModeling GitHub organization
2. Bug with reading dss data (bug with writing data was solved)
3. HecDss7 branch vs WRIMSDevOps branch
4. @rma-tom discussion of WRIMS-core Regression/Comparison Test Design
5. Discuss use of artifact repository for jars and dlls bundled with CalSim studies

Open Action Items:

• @rma-tom to submit prototyping of comparison tests into Feature/wrims-devops. Submit prototyping of comparison testing #54
• @rma-tom to set-up artifact repository for hosting artifacts and watersheds required for automated testing. Set-up artifact repository to host artifacts and watersheds required for automated tests for WRIMS DevOps #53
• Coordinate with Hao Xie for manual steps in creating WRIMS installer and patch artifacts. This is tracked as Demonstrate Eclipse IDE steps and Manual steps involved with creating both the installer and patch artifacts (by Hao) #29.
• Apply copyright files (license and source) in the wrims repository using Eclipse Public License (EPL) version 2 or 3. This is tracked as Apply copyright file for the repository #16.
• Remove vista libraries from WRIMS. This is tracked as Remove vista library from WRIMS #33.

Discussions:

• Discussion was held regarding branch security for the CentralValleyModeling GitHub organization.
  • This discussion was originally brought up by Raymond Hoang when he inquired regarding recommendations for branch security for CentralValleyModeling organization in GitHub.
  • Security is in layers in GitHub. RMA has focused on establishing security on the Feature/wrims-devops branch, which is the lowest layer. The details on the established branch security was forwarded in a previous email to DWR.
  • Hao is happy with the current set-up of branch security.
  • If Raymond has any recommendations, he can communicate with RMA.
• Hao reported that when using the read function in the HecLib 7 library to read HEC-DSS version 6 data, reading the data is accurate but slow. This is tracked as HecLib version 7 reading DSS files takes a significant amount of time #52.
  • Adam reported that Karl Tarbet (HEC) thinks this may be a bug and something that he is currently looking into.
  • The issue with writing HEC-DSS version 6 data has been resolved.
• Yiwei asked if the issues with the current version of HecLib version 7 should be communicated with the team at DWR on python work that calls on functions from HecLib.
  • Hao recommends that Yiwei determine which version of HecLib is being used for the python work. The previously discussed issue is not present in HecLib version 6.
• HecDss7 branch vs WRIMSDevOps branch was discussed.
  • Hao wanted to know the difficulties of integrating his changes into the Feature/HecDSS7 branch into the Feature/wrims-devops branch.
  • It is easier to merge changes from one branch instead of 2 branches. There are multiple steps to merge a change from Feature/HecDSS7 to Feature/wrims-devops. The intention is to merge these two branches then that will become the new master branch.
  • After the remaining issue with reading DSS data in HecLib version 7 is resolved, the Feature/HecDSS7 branch will be merged with the Feature/wrims-devops branch.
  • Adam recommends re-evaluating the merging of branches after the remaining issue with HecLib version 7 is resolved.
• Tom discussed the WRIMS-core Regression/Comparison Test Design that he prepared.
  • Tom, Peter, Adam, and Deziree are collaborating on the comparison design currently documented in a Word document. This will eventually be shared with the entire team as a GitHub discussion.
  • Tom summarized that the watersheds from CalSim and CalLite will be stored in a artifact repository, then these watershed can be used to execute multiple runs. The results can then be compared from the runs.
  • The intention with the regression tests is to have a baseline to compare results against. These new results are from builds created through the Gradle system. A tool can then be used to provide the comparison results.
  • JUnit was one tool considered. However, Peter thinks this is the wrong tool as this only provides a pass/fail result instead of a full report on the comparison.
  • EPPT is also considered as a comparison tool to use as this has many ways of doing comparisons. However, it may get out of date.
  • Hao and Nazrul recommends using the CalLite Report Tool, which is already included in the wrims2.jar. This may require a template file and Hao can assist with preparing that file if this tool is used.
  • Hao is concerned with how precision can be handled when the intention is to create a more systematic comparison to determine if the results of two studies are the same or not.
    • Tom uses the HEC-DSSVue's comparison feature where precision may be set. This feature also allows for filtering to specific DSS pathname patterns as necessary. This code exists in HEC's monolith and may pulled into the Feature/wrims-devops.
    • Peter also added that the EPPT may be updated to incorporate the pathname filtering and additional connectivity pieces.
  • Tom will soon submit a pull request with his prototyping for this work. This is tracked in Submit prototyping of comparison testing #54
• Tom discussed Intellectual Property concerns regarding 3rd party jars and dlls bundled with CalSim and CalLite studies.
  • Tom recommended using an artifact repository to hold the 3rd party jars so that these are not accessible to other parties.
  • These jars and dlls are required for making runtime work for automated tests.
  • Tom previously recommended that Hao remove some of the 3rd party libraries and replacing them with publicly available libraries instead. Hao will address these after the remaining issue with HecDSS7 has been addressed.
  • Hao determined that the dlls are DWR libraries.
  • Adam also recommended that a username and password can be required to control access to artifacts on an artifact repository. Currently, these are source code in the repository and could be hosted as artifacts instead.
  • Peter added that in the meantime, an artifact repository may be established to hold the watersheds and artifacts and the artifact repository may be changed later on.
  • Hao approved that Tom can move forward with creating an artifact repository for the watersheds and artifacts as needed. This is tracked as Set-up artifact repository to host artifacts and watersheds required for automated tests for WRIMS DevOps #53.
• Yiwei requested assistance from RMA to address an issue with the CalLite GeoUI.
  • Yiwei will coordinate with Nazrul and Hao regarding work to begin addressing CalLite issues.
  • Peter confirmed that resources are available to work on issues related to CalLite. Included in the group are Adam and Deziree. Additional resources may be brought in as needed.
• Hao asked if a demonstration for the manual steps in creating the WRIMS installer and patch artifacts is still needed. This is tracked as Demonstrate Eclipse IDE steps and Manual steps involved with creating both the installer and patch artifacts (by Hao) #29.
  • Because there are specific manual steps necessary to create the WRIMS installer, the demonstration is still necessary so that the procedure can be captured for automation.
  • Hao will be notified when RMA is ready for the demonstration.

Closed Action Items:

• Have a discussion on branch security for the CentralValleyModeling GitHub organization.
• @hxiedwr will write a unit test for exposing the bad behavior in the HecLib 7 producing HEC-DSS version 6 files with data that are generated at 24:00 instead of 00:00 and causing an offset. Create unit test that exposes issue with DSS data being generated at the incorrect date and time #44
• @rma-tom will merge his updates for the README.md files into the Feature/HecDSS7 branch. Tracked as Document descriptions for each folder in the wrims2 repository #34.

[hxiedwr]

The dss reading has two methods:

1. using cache read method. It is fast but misses data in certain period after retrieving HecLib 6 format dss data with HecLib 7 library.
2. using hecdss get method. it is slow but no missing data after retrieving HecLib 6 format dss data with HecLib 7 library.
   We expect method 1 to be fixed/improved. So retrieving data is both fast and accurate.

[adamkorynta]

A quick update: @hxiedwr Karl at HEC has created a unit test for the time window behavior: https://github.com/HydrologicEngineeringCenter/hec-dss/blob/bug/no-time-window/test/Dss-C/source/testNoDates.c

We also verified that the bug is even with retrieving DSS 7 format data with Heclib 7 library, not just DSS 6 format.