diff --git a/docs/conversion/create-snapshots.md b/docs/conversion/create-snapshots.md index dddbf5a..78815e6 100644 --- a/docs/conversion/create-snapshots.md +++ b/docs/conversion/create-snapshots.md @@ -1,5 +1,11 @@ # 6 - Create snapshots of CAM +## CAM snapshot run tips + +1. Try to use the lowest resolution grid possible in order to keep the snapshot file size small (e.g. `ne3pg3_ne3pg3_mg37`). +1. Run with debug flags on (`./xmlchange DEBUG=True`), as this way errors that can occur when you bring the CCPP-ized code back into CAM are easier to catch. Make sure ALL CAM and CAM-SIMA runs have debug on. +1. You should generate at least one set of snapshot files using GNU on derecho with DEBUG=TRUE, as these will be saved for use in the CAM-SIMA physics testbed to prevent unexpected answer changes. + ## Configure and set-up CAM snapshot Make a normal **CAM** (not CAM-SIMA) run using cam_snapshot to capture before and after files using a compset which exercises the parameterization being converted. @@ -9,53 +15,8 @@ Make a normal **CAM** (not CAM-SIMA) run using cam_snapshot to capture before an - To find the snapshot name, look in your source code checkout in `bld/namelist_files/namelist_definition.xml`. - Inside this file, find the “cam_take_snapshot_before” definition. In this definition, it lists all possible parameterizations which can have a snapshot file created around. - If your parameterization is there, that will be your `cam_take_snapshot_before` and `_after` values in your `user_nl_cam` - - If it's not there, you'll add `user_set` snapshot calls to `physpkg.F90` in the following way: - -For `tphysbc` in `physics/cam7/physpkg.F90`: -``` -if (trim(cam_take_snapshot_before) == "user_set") then - call cam_snapshot_all_outfld_tphysbc(cam_snapshot_before_num, state, tend, cam_in, cam_out, pbuf, & - flx_heat, cmfmc, cmfcme, zdu, rliq, rice, dlf, dlf2, rliq2, det_s, det_ice, net_flx) -end if - - - -if ( (trim(cam_take_snapshot_after) == "user_set") .and. & - (trim(cam_take_snapshot_before) == trim(cam_take_snapshot_after))) then - call cam_snapshot_ptend_outfld(ptend, lchnk) -end if - - - -if (trim(cam_take_snapshot_after) == "user_set") then - call cam_snapshot_all_outfld_tphysbc(cam_snapshot_after_num, state, tend, cam_in, cam_out, pbuf, & - flx_heat, cmfmc, cmfcme, zdu, rliq, rice, dlf, dlf2, rliq2, det_s, det_ice, net_flx) -end if -``` - -For `tphysac`: -``` -if (trim(cam_take_snapshot_before) == "user_set") then - call cam_snapshot_all_outfld_tphysac(cam_snapshot_before_num, state, tend, cam_in, cam_out, pbuf, & - flx_heat, cmfmc, cmfcme, zdu, rliq, rice, dlf, dlf2, rliq2, det_s, det_ice, net_flx) -end if - - + - If it's not there, you'll add `user_set` snapshot calls to `physpkg.F90` per the instructions [here](#user-set-snapshot-calls) -if ( (trim(cam_take_snapshot_after) == "user_set") .and. & - (trim(cam_take_snapshot_before) == trim(cam_take_snapshot_after))) then - call cam_snapshot_ptend_outfld(ptend, lchnk) -end if - - - -if (trim(cam_take_snapshot_after) == "user_set") then - call cam_snapshot_all_outfld_tphysac(cam_snapshot_after_num, state, tend, cam_in, cam_out, pbuf, & - flx_heat, cmfmc, cmfcme, zdu, rliq, rice, dlf, dlf2, rliq2, det_s, det_ice, net_flx) -end if -``` -!!! Note - The above calls may vary if your physpkg.F90 exists in a different directory within `$CAM/src/physics` (not `cam7`) - To enable cam_snapshot, add the following to your user_nl_cam file: @@ -78,7 +39,7 @@ ndens = 2,2,2,2,2,1,1 - The run must be at least 2 time steps in order to work in subsequent steps. We recommend 3 to at most 9 time steps. (Note a 3 timestep run using a 2 degree FV grid can produce snapshot files with sizes over 4 Gb each). - To change the number of timesteps to 3 use `./xmlchange STOP_OPTION=nsteps` and `./xmlchange STOP_N=3` -!!! Warning +!!! Note "Coupling frequency" With short NUOPC runs, you may need to change ROF_NCPL and/or GLC_NCPL to the same size at ATM_NCPL to get them to run properly (`./xmlchange ROF_NCPL=\$ATM_NCPL` and `./xmlchange GLC_NCPL=\$ATM_NCPL`) ## Create CAM snapshots @@ -113,11 +74,106 @@ On **izumi**, save your converted snapshot files in: /project/amp02/cam_snapshot_files/ ``` -## CAM snapshot run tips +Once you have created your snapshot files, proceed to [7 - Run CAM-SIMA](run-cam-sima.md) -1. Try to use the lowest resolution grid possible in order to keep the snapshot file size small (e.g. `ne3pg3_ne3pg3_mg37`). -1. We generally recommend running with debug flags on (`./xmlchange DEBUG=True`), as this way errors that can occur when you bring the CCPP-ized code back into CAM are easier to catch. - - However, if you decide not to, then do make sure that all of your other CAM runs also have debug flags off, as otherwise you can get differences in the results that are solely due to compiler settings, and not your code changes. -1. You should generate at least one set of snapshot files using GNU on derecho with DEBUG=TRUE, as these will be saved for use in the CAM-SIMA physics testbed to prevent unexpected answer changes. +## "user set" snapshot calls +Depending on where your scheme is called within CAM physics (in CAM6 or CAM7 physpkg and in tphysac or tphysbc), choose a code snippet below. -Once you have created your snapshot files, proceed to [7 - Run CAM-SIMA](run-cam-sima.md) +!!! Warning "Snapshot call changes" + If the snapshot calls change in CAM, this section may not be updated immediately. If you're running into issues, mirror an existing set of snapshot calls in the physpkg.F90 routine you are working in. + +### CAM7 tphysbc + +In `$CAM/src/physics/cam7/physpkg.F90`: + +``` +if (trim(cam_take_snapshot_before) == "user_set") then + call cam_snapshot_all_outfld_tphysbc(cam_snapshot_before_num, state, tend, cam_in, cam_out, pbuf, & + cmfmc, cmfcme, zdu, rliq, rice, dlf, dlf2, rliq2, net_flx) +end if + + + +if ( (trim(cam_take_snapshot_after) == "user_set") .and. & + (trim(cam_take_snapshot_before) == trim(cam_take_snapshot_after))) then + call cam_snapshot_ptend_outfld(ptend, lchnk) +end if + + + +if (trim(cam_take_snapshot_after) == "user_set") then + call cam_snapshot_all_outfld_tphysbc(cam_snapshot_after_num, state, tend, cam_in, cam_out, pbuf, & + cmfmc, cmfcme, zdu, rliq, rice, dlf, dlf2, rliq2, net_flx) +end if +``` + +### CAM7 tphysac +In `$CAM/src/physics/cam7/physpkg.F90` + +``` +if (trim(cam_take_snapshot_before) == "user_set") then + call cam_snapshot_all_outfld_tphysac(cam_snapshot_before_num, state, tend, cam_in, cam_out, pbuf,& + fh2o, surfric, obklen, flx_heat, cmfmc, dlf, det_s, det_ice, net_flx) +end if + + + +if ( (trim(cam_take_snapshot_after) == "user_set") .and. & + (trim(cam_take_snapshot_before) == trim(cam_take_snapshot_after))) then + call cam_snapshot_ptend_outfld(ptend, lchnk) +end if + + + +if (trim(cam_take_snapshot_after) == "user_set") then + call cam_snapshot_all_outfld_tphysac(cam_snapshot_after_num, state, tend, cam_in, cam_out, pbuf,& + fh2o, surfric, obklen, flx_heat, cmfmc, dlf, det_s, det_ice, net_flx) +end if +``` + +### CAM6 tphysbc +In `$CAM/src/physics/cam/physpkg.F90`: +``` +if (trim(cam_take_snapshot_before) == "user_set") then + call cam_snapshot_all_outfld_tphysbc(cam_snapshot_before_num, state, tend, cam_in, cam_out, pbuf, & + flx_heat, cmfmc, cmfcme, zdu, rliq, rice, dlf, dlf2, rliq2, det_s, det_ice, net_flx) +end if + + + +if ( (trim(cam_take_snapshot_after) == "user_set") .and. & + (trim(cam_take_snapshot_before) == trim(cam_take_snapshot_after))) then + call cam_snapshot_ptend_outfld(ptend, lchnk) +end if + + + +if (trim(cam_take_snapshot_after) == "user_set") then + call cam_snapshot_all_outfld_tphysbc(cam_snapshot_after_num, state, tend, cam_in, cam_out, pbuf, & + flx_heat, cmfmc, cmfcme, zdu, rliq, rice, dlf, dlf2, rliq2, det_s, det_ice, net_flx) +end if +``` + +### CAM6 tphysac +In `$CAM/src/physics/cam/physpkg.F90`: + +``` +if (trim(cam_take_snapshot_before) == "user_set") then + call cam_snapshot_all_outfld_tphysac(cam_snapshot_before_num, state, tend, cam_in, cam_out, pbuf,& + fh2o, surfric, obklen, flx_heat) +end if + + + +if ( (trim(cam_take_snapshot_after) == "user_set") .and. & + (trim(cam_take_snapshot_before) == trim(cam_take_snapshot_after))) then + call cam_snapshot_ptend_outfld(ptend, lchnk) +end if + + + +if (trim(cam_take_snapshot_after) == "user_set") then + call cam_snapshot_all_outfld_tphysac(cam_snapshot_after_num, state, tend, cam_in, cam_out, pbuf,& + fh2o, surfric, obklen, flx_heat) +end if +```