-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Workflow code that run topmodel within ngen Framework #42
Changes from 18 commits
8216e4c
470f301
92ce260
5ad8842
bd5aa3e
497b924
0b90dda
88fbf3a
e5b4f0b
3d0f461
3351ef6
25958ce
d8baa04
6101462
6e2493c
a036671
25989b1
aa2c38a
37c5044
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,119 @@ | ||
# Test ngen-cfe integration | ||
|
||
name: Ngen Integration Tests | ||
|
||
# Controls when the action will run. | ||
on: | ||
push: | ||
branches: [ master, dev, notreal ] | ||
pull_request: | ||
branches: [ master, dev, notreal ] | ||
workflow_dispatch: | ||
|
||
env: | ||
# Obtained from https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources | ||
# TODO: add something later that can check the runners for hyperthreading | ||
LINUX_NUM_PROC_CORES: 2 | ||
MACOS_NUM_PROC_CORES: 3 | ||
ASAN_OPTIONS: detect_leaks=false | ||
|
||
# A workflow run is made up of one or more jobs that can run sequentially or in parallel | ||
jobs: | ||
# Run general unit tests in linux environment | ||
test_surfacebmi_plus_cfe: | ||
# The type of runner that the job will run on | ||
strategy: | ||
matrix: | ||
os: [ubuntu-latest, macos-latest] | ||
fail-fast: false | ||
runs-on: ${{ matrix.os }} | ||
|
||
# Steps represent a sequence of tasks that will be executed as part of the job | ||
steps: | ||
# Checkout and save the topmodel to a temporary directory | ||
- name: Checkout the commit | ||
uses: actions/checkout@v4 | ||
|
||
- name: Save Topmodel to a Temp Directory | ||
run: | | ||
# Move files to a temporary directory | ||
mkdir ${{runner.temp}}/topmodel | ||
mv ./* ${{runner.temp}}/topmodel | ||
|
||
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it | ||
- name: checkout actions in another repo | ||
uses: actions/checkout@v4 | ||
with: | ||
repository: noaa-owp/ngen | ||
|
||
- name: Build Surfacebmi | ||
id: submod_build_1 | ||
uses: ./.github/actions/ngen-submod-build | ||
with: | ||
mod-dir: " extern/noah-owp-modular/" | ||
targets: "surfacebmi" | ||
|
||
- name: Build ISO C Fortran BMI | ||
id: submod_build_2 | ||
uses: ./.github/actions/ngen-submod-build | ||
with: | ||
mod-dir: "extern/iso_c_fortran_bmi/" | ||
|
||
- name: Build CFE | ||
id: submod_build_3 | ||
uses: ./.github/actions/ngen-submod-build | ||
with: | ||
mod-dir: "extern/cfe/" | ||
targets: "cfebmi" | ||
|
||
- name: Build PET | ||
id: submod_build_5 | ||
uses: ./.github/actions/ngen-submod-build | ||
with: | ||
mod-dir: "extern/evapotranspiration/evapotranspiration" | ||
targets: "petbmi" | ||
|
||
- name: Build SLoTH | ||
id: submod_build_6 | ||
uses: ./.github/actions/ngen-submod-build | ||
with: | ||
mod-dir: "extern/sloth/" | ||
targets: "slothmodel" | ||
|
||
# Build Ngen and save some files | ||
- name: Build Ngen | ||
id: ngen_id1 | ||
uses: ./.github/actions/ngen-build | ||
with: | ||
targets: "ngen" | ||
build-cores: ${{ env.LINUX_NUM_PROC_CORES }} | ||
#is this required for this test? | ||
bmi_c: 'ON' | ||
bmi_fortran: 'ON' | ||
timeout-minutes: 15 | ||
|
||
- name: Prepare to Build Topmodel Library for Ngen | ||
run: | | ||
# Move files to appropriate directory to build topmodel library | ||
rm -rf extern/topmodel/topmodel/* | ||
mv ${{runner.temp}}/topmodel/* extern/topmodel/topmodel | ||
|
||
- name: Build Topmodel | ||
id: submod_build_4 | ||
uses: ./.github/actions/ngen-submod-build | ||
with: | ||
mod-dir: " extern/topmodel/" | ||
targets: "topmodelbmi" | ||
|
||
- name: Run Ngen Test | ||
run: | | ||
# Run ngen with topmodel with CFE | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Again, this realization couples Is a It's probably best to throw There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Tha comment is incorrect. I think it is either topmodel or CFE. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could try to see if you can find your realization file coupling PET with Topmodel? That would save me time if there are intricacies involved. Thanks. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
mv ${{ steps.ngen_id1.outputs.build-dir }} ./ngen-build/ | ||
inputfile='data/example_bmi_multi_realization_config.json' | ||
./ngen-build/ngen ./data/catchment_data.geojson "cat-27" ./data/nexus_data.geojson "nex-26" $inputfile | ||
|
||
- name: Run surfacebmi, topmodelbmi | ||
run: | | ||
cp extern/topmodel/topmodel/data/*.dat data | ||
inputfile='extern/topmodel/topmodel/data/realization_config_topmodel_ngenCI.json' | ||
./ngen-build/ngen ./data/catchment_data.geojson "cat-27" ./data/nexus_data.geojson "nex-26" $inputfile |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
{ | ||
"global": { | ||
"formulations": [ | ||
{ | ||
"name": "bmi_multi", | ||
"params": { | ||
"model_type_name": "bmi_multi_noahowp_cfe", | ||
"forcing_file": "", | ||
"init_config": "", | ||
"allow_exceed_end_time": true, | ||
"main_output_variable": "Qout", | ||
"modules": [ | ||
{ | ||
"name": "bmi_c++", | ||
"params": { | ||
"model_type_name": "bmi_c++_sloth", | ||
"library_file": "./extern/sloth/cmake_build/libslothmodel", | ||
"init_config": "/dev/null", | ||
"allow_exceed_end_time": true, | ||
"main_output_variable": "z", | ||
"uses_forcing_file": false, | ||
"model_params": { | ||
"sloth_ice_fraction_schaake(1,double,m,node)": 0.0, | ||
"sloth_ice_fraction_xinan(1,double,1,node)": 0.0, | ||
"sloth_smp(1,double,1,node)": 0.0 | ||
} | ||
} | ||
}, | ||
{ | ||
"name": "bmi_fortran", | ||
"params": { | ||
"model_type_name": "bmi_fortran_noahowp", | ||
"library_file": "./extern/noah-owp-modular/cmake_build/libsurfacebmi", | ||
"forcing_file": "", | ||
"init_config": "./data/bmi/fortran/noah-owp-modular-init-{{id}}.namelist.input", | ||
"allow_exceed_end_time": true, | ||
"main_output_variable": "QINSUR", | ||
"variables_names_map": { | ||
"PRCPNONC": "atmosphere_water__liquid_equivalent_precipitation_rate", | ||
"Q2": "atmosphere_air_water~vapor__relative_saturation", | ||
"SFCTMP": "land_surface_air__temperature", | ||
"UU": "land_surface_wind__x_component_of_velocity", | ||
"VV": "land_surface_wind__y_component_of_velocity", | ||
"LWDN": "land_surface_radiation~incoming~longwave__energy_flux", | ||
"SOLDN": "land_surface_radiation~incoming~shortwave__energy_flux", | ||
"SFCPRS": "land_surface_air__pressure" | ||
}, | ||
"uses_forcing_file": false | ||
} | ||
}, | ||
{ | ||
"name": "bmi_c", | ||
"params": { | ||
"model_type_name": "bmi_c_topmodel", | ||
"library_file": "./extern/topmodel/cmake_build/libtopmodelbmi.so", | ||
"forcing_file": "", | ||
"init_config": "./extern/topmodel/topmodel/data/topmod.run", | ||
"allow_exceed_end_time": true, | ||
"main_output_variable": "Qout", | ||
"registration_function": "register_bmi_topmodel", | ||
"variables_names_map": { | ||
"water_potential_evaporation_flux": "EVAPOTRANS", | ||
"atmosphere_water__liquid_equivalent_precipitation_rate": "QINSUR" | ||
}, | ||
"uses_forcing_file": false | ||
} | ||
} | ||
], | ||
"uses_forcing_file": false | ||
} | ||
} | ||
], | ||
"forcing": { | ||
"file_pattern": ".*{{id}}.*..csv", | ||
"path": "./data/forcing/", | ||
"provider": "CsvPerFeature" | ||
} | ||
}, | ||
"time": { | ||
"start_time": "2015-12-01 00:00:00", | ||
"end_time": "2015-12-30 23:00:00", | ||
"output_interval": 3600 | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see
PET
being used in any of thengen
runs here.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, it is not so can be removed.