Add namelist control of PIO options #74
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Fix CESMCOUPLED compile problem in icepack
… bug (CICE-Consortium#824) * Update global reduction implementation to improve performance, fix VP bug This was mainly done for situations like VP that need a fast global sum. The VP global sum is still slightly faster than the one computed in the infrastructure, so kept that implementation. Found a bug in the workspace_y calculation in VP that was fixed. Also found that the haloupdate call as part of the precondition step generally improves VP performance, so removed option to NOT call the haloupdate there. Separately, fixed a bug in the tripoleT global sum implementation, added a tripoleT global sum unit test, and resynced ice_exit.F90, ice_reprosum.F90, and ice_global_reductions.F90 between serial and mpi versions. - Refactor global sums to improve performance, move if checks outside do loops - Fix bug in tripoleT global sums, tripole seam masking - Update VP solver, use local global sum more often - Update VP solver, fix bug in workspace_y calculation - Update VP solver, always call haloupdate during precondition - Refactor ice_exit.F90 and sync serial and mpi versions - Sync ice_reprosum.F90 between serial and mpi versions - Update sumchk unit test to handle grids better - Add tripoleT sumchk test * Update VP global sum to exclude local implementation with tripole grids
…#827) * Add functionality to change hist_avg for each stream * Fix some documentation * Try to fix sphinx problem * Fix hist_avg documentation * Add some metadata changes to time and time_bounds
Remove trailing blanks via automated tool in some Fortran files
* Fix for mesh check in CESM driver * Slightly different way to evaluate longitude difference * Slightly different way to evaluate longitude difference * Put the abs inside the mod * Add abort calls back in
* update readthedocs.yaml, turn on pdf * update readthedocs.yaml, turn on pdf * update readthedocs.yaml, turn on pdf * update readthedocs.yaml, turn on pdf
* Add option to change location in interval of time axis * Only use hist_time_axis when hist_avg is true * Add more comments and information in the documentation * Add a check on hist_time_axis as well as a global attribute * Abort if hist_time_axis is not set correctly.
Remove deprecated COREII LYq forcing Remove deprecated print_points_state Update links in rst documentation to point to main, not master
* updating paths for local nrlssc builds * Add jra55do forcing option * Updated env.nrlssc_gnu for new local directory structure * Added JRA55do to file names. Added comments for each variable name at top of JRA55do_???_files subroutine * Make JRA55 forcing to use common subroutines. Search atm_data_type for specific cases * remove extraneous 'i' variable in JRA55_files * Changed JRA55 filename JRA55_grid instead of grid at end of filename * Add jra55do tests to base_suite and quick_suite. This is done via set_nml options. * Update forcing implementation to provide a little more flexibility for JRA55, JRA55do, and ncar bulk atm forcing files. * Update documentation * update Onyx port * Update forcing documentation Initial port to derecho_intel * clean up blank spaces --------- Co-authored-by: daveh150 <david.hebert@nrlssc.navy.mil>
…tium#846) - fix optional arguments issues - fix hsn_new(1) bug Update optargs unit test, add new test cases Add opticep unit test, to test CICE calls to Icepack without optional arguments. Add new comparison option to comparelog.csh to compare a unit test with a standard CICE test. Update unittest_suite Update documentation about optional arguments and unit tests
…al remapping (CICE-Consortium#833) * Use same method whether l_fixed_area=T or F to compute masks for edge fluxes * Corrected typo in comment * Cosmetic (indentation) change in ice_transport_remap.F90 * Set l_fixed_area value depending of grid type * Modifs to the doc for l_fixed_area * Use umask for uvel,vvel initialization for boxslotcyl and change grid avg type from S to A in init_state * Temporary changes before next PR: l_fixed_area=F for B and C grid * Temporary changes before next PR: remove paragraph in the doc * Small modifs: l_fixed_area and grid_ice are defined in module ice_transport_remap
…Consortium#853) * test ghactions * update master to main in github actions
* Replace tr_snow flag with snwredist, snwgrain in some places (tr_snow is still used more generally). Fix intent(out) compile issue in ice_read_write.F90. Replace badger with chicoma machine files. * update icepack to 86cae16d1b7c4c4f8 --------- Co-authored-by: apcraig <anthony.p.craig@gmail.com>
* Deprecate zsalinity, mostly with ifdef and comments first for testing * Deprecate zsalinity, remove code * Add warning message for deprecated zsalinity * Update Icepack to #f5e093f5148554674 (deprecate zsalinity)
…ICE-Consortium#850) * Extend restart output controls, provide multiple streams for possible output frequencies. Convert dumpfreq, dumpfreq_n, dumpfreq_base to arrays. Modify histfreq_base to make it an array as well. Now each history stream can have it's own base time (init or zero). Update documentation. * Clean up implementation and documentation * Update PR to check github actions
…ICE-Consortium#856) * merge latest master (NOAA-EMC#4) * Isotopes for CICE (CICE-Consortium#423) Co-authored-by: apcraig <anthony.p.craig@gmail.com> Co-authored-by: David Bailey <dbailey@ucar.edu> Co-authored-by: Elizabeth Hunke <eclare@lanl.gov> * updated orbital calculations needed for cesm * fixed problems in updated orbital calculations needed for cesm * update CICE6 to support coupling with UFS * put in changes so that both ufsatm and cesm requirements for potential temperature and density are satisfied * Convergence on ustar for CICE. (CICE-Consortium#452) (NOAA-EMC#5) * Add atmiter_conv to CICE * Add documentation * trigger build the docs Co-authored-by: David A. Bailey <dbailey@ucar.edu> * update icepack submodule * Revert "update icepack submodule" This reverts commit e70d1ab. * update comp_ice.backend with temporary ice_timers fix * Fix threading problem in init_bgc * Fix additional OMP problems * changes for coldstart running * Move the forapps directory * remove cesmcoupled ifdefs * Fix logging issues for NUOPC * removal of many cpp-ifdefs * fix compile errors * fixes to get cesm working * fixed white space issue * Add restart_coszen namelist option * update icepack submodule * change Orion to orion in backend remove duplicate print lines from ice_transport_driver * add -link_mpi=dbg to debug flags (NOAA-EMC#8) * cice6 compile (NOAA-EMC#6) * enable debug build. fix to remove errors * fix an error in comp_ice.backend.libcice * change Orion to orion for machine identification * changes for consistency w/ current emc-cice5 (NOAA-EMC#13) Update to emc/develop fork to current CICE consortium Co-authored-by: David A. Bailey <dbailey@ucar.edu> Co-authored-by: Tony Craig <apcraig@users.noreply.github.com> Co-authored-by: Elizabeth Hunke <eclare@lanl.gov> Co-authored-by: Mariana Vertenstein <mvertens@ucar.edu> Co-authored-by: apcraig <anthony.p.craig@gmail.com> Co-authored-by: Philippe Blain <levraiphilippeblain@gmail.com> * Fixcommit (NOAA-EMC#14) Align commit history between emc/develop and cice-consortium/master * Update CICE6 for integration to S2S * add wcoss_dell_p3 compiler macro * update to icepack w/ debug fix * replace SITE with MACHINE_ID * update compile scripts * Support TACC stampede (NOAA-EMC#19) * update icepack * add ice_dyn_vp module to CICE_InitMod * update gitmodules, update icepack * Update CICE to consortium master (NOAA-EMC#23) updates include: * deprecate upwind advection (CICE-Consortium#508) * add implicit VP solver (CICE-Consortium#491) * update icepack * switch icepack branches * update to icepack master but set abort flag in ITD routine to false * update icepack * Update CICE to latest Consortium master (NOAA-EMC#26) update CICE and Icepack * changes the criteria for aborting ice for thermo-conservation errors * updates the time manager * fixes two bugs in ice_therm_mushy * updates Icepack to Consortium master w/ flip of abort flag for troublesome IC cases * add cice changes for zlvs (NOAA-EMC#29) * update icepack and pointer * update icepack and revert gitmodules * Fix history features - Fix bug in history time axis when sec_init is not zero. - Fix issue with time_beg and time_end uninitialized values. - Add support for averaging with histfreq='1' by allowing histfreq_n to be any value in that case. Extend and clean up construct_filename for history files. More could be done, but wanted to preserve backwards compatibility. - Add new calendar_sec2hms to converts daily seconds to hh:mm:ss. Update the calchk calendar unit tester to check this method - Remove abort test in bcstchk, this was just causing problems in regression testing - Remove known problems documentation about problems writing when istep=1. This issue does not exist anymore with the updated time manager. - Add new tests with hist_avg = false. Add set_nml.histinst. * revert set_nml.histall * fix implementation error * update model log output in ice_init * Fix QC issues - Add netcdf ststus checks and aborts in ice_read_write.F90 - Check for end of file when reading records in ice_read_write.F90 for ice_read_nc methods - Update set_nml.qc to better specify the test, turn off leap years since we're cycling 2005 data - Add check in c ice.t-test.py to make sure there is at least 1825 files, 5 years of data - Add QC run to base_suite.ts to verify qc runs to completion and possibility to use those results directly for QC validation - Clean up error messages and some indentation in ice_read_write.F90 * Update testing - Add prod suite including 10 year gx1prod and qc test - Update unit test compare scripts * update documentation * reset calchk to 100000 years * update evp1d test * update icepack * update icepack * add memory profiling (NOAA-EMC#36) * add profile_memory calls to CICE cap * update icepack * fix rhoa when lowest_temp is 0.0 * provide default value for rhoa when imported temp_height_lowest (Tair) is 0.0 * resolves seg fault when frac_grid=false and do_ca=true * update icepack submodule * Update CICE for latest Consortium master (NOAA-EMC#38) * Implement advanced snow physics in icepack and CICE * Fix time-stamping of CICE history files * Fix CICE history file precision * Use CICE-Consortium/Icepack master (NOAA-EMC#40) * switch to icepack master at consortium * recreate cap update branch (NOAA-EMC#42) * add debug_model feature * add required variables and calls for tr_snow * remove 2 extraneous lines * remove two log print lines that were removed prior to merge of driver updates to consortium * duplicate gitmodule style for icepack * Update CICE to latest Consortium/main (NOAA-EMC#45) * Update CICE to Consortium/main (NOAA-EMC#48) Update OpenMP directives as needed including validation via new omp_suite. Fixed OpenMP in dynamics. Refactored eap puny/pi lookups to improve scalar performance Update Tsfc implementation to make sure land blocks don't set Tsfc to freezing temp Update for sea bed stress calculations * fix comment, fix env for orion and hera * replace save_init with step_prep in CICE_RunMod * fixes for cgrid repro * remove added haloupdates * baselines pass with these extra halo updates removed * change F->S for ocean velocities and tilts * fix debug failure when grid_ice=C * compiling in debug mode using -init=snan,arrays requires initialization of variables * respond to review comments * remove inserted whitespace for uvelE,N and vvelE,N * Add wave-cice coupling; update to Consortium main (NOAA-EMC#51) * add wave-ice fields * initialize aicen_init, which turns up as NaN in calc of floediam export * add call to icepack_init_wave to initialize wavefreq and dwavefreq * update to latest consortium main (PR 752) * add initializationsin ice_state * initialize vsnon/vsnon_init and vicen/vicen_init * Update CICE (NOAA-EMC#54) * update to include recent PRs to Consortium/main * fix for nudiag_set allow nudiag_set to be available outside of cesm; may prefer to fix in coupling interface * Update CICE for latest Consortium/main (NOAA-EMC#56) * add run time info * change real(8) to real(dbl)kind) * fix syntax * fix write unit * use cice_wrapper for ufs timer functionality * add elapsed model time for logtime * tidy up the wrapper * fix case for 'time since' at the first advance * add timer and forecast log * write timer values to timer log, not nu_diag * write log.ice.fXXX * only one time is needed * modify message written for log.ice.fXXX * change info in fXXX log file * Update CICE from Consortium/main (NOAA-EMC#62) * Fix CESMCOUPLED compile issue in icepack. (CICE-Consortium#823) * Update global reduction implementation to improve performance, fix VP bug (CICE-Consortium#824) * Update VP global sum to exclude local implementation with tripole grids * Add functionality to change hist_avg for each stream (CICE-Consortium#827) * Update Icepack to #6703bc533c968 May 22, 2023 (CICE-Consortium#829) * Fix for mesh check in CESM driver (CICE-Consortium#830) * Namelist option for time axis position. (CICE-Consortium#839) * reset timer after Advance to retrieve "wait time" * add logical control for enabling runtime info * remove zsal items from cap * fix typo --------- Co-authored-by: apcraig <anthony.p.craig@gmail.com> Co-authored-by: David Bailey <dbailey@ucar.edu> Co-authored-by: Elizabeth Hunke <eclare@lanl.gov> Co-authored-by: Mariana Vertenstein <mvertens@ucar.edu> Co-authored-by: Minsuk Ji <57227195+MinsukJi-NOAA@users.noreply.github.com> Co-authored-by: Tony Craig <apcraig@users.noreply.github.com> Co-authored-by: Philippe Blain <levraiphilippeblain@gmail.com> Co-authored-by: Jun.Wang <Jun.Wang@noaa.gov>
Includes thin ice enthalpy fix, not bit-for-bit.
* Update cheyenne and derecho ports cheyenne_intel updated to intel/19/1/1, mpt/2.25 cheyenne_gnu updated to gnu/8.3.0, mpt/2.25 cheyenne_pgi updated to pgi/19.9, mpt/2.22 derecho_intel minor updates derecho_intelclassic added derecho_inteloneapi added (not working) derecho_gnu added derecho_cray added derecho_nvhpc added cheyenne_pgi changed answers derecho_inteloneapi is not working, compiler issues fixes automated qc testing on cheyenne * Update permissions on env.chicoma_intel
Update License and Copyright Update Icepack for version/copyright
* Modified doc to specify that l_fixed_area is T for C-grid * Initial modifs to calc areafact based on linear interpolation of left and rigth values * put back l_fixed_area = .true. for C-grid * added temporary comments for PR review * Modified areafac calc for case 1 and case 2 * Corrected minor compilation issues * Corrected conditions for case 1 to make sure areas add up * Small modif in l_fixed_area section to ensure only one condition is true * Modified conditions in locate triangle to be consistent with previous changes for case 1 * Use other edge areafac_c for TL, BL, TR and BR triangles * Some comments removed * Fixed out of bounds areafac_ce and now use earea and narea * Replaced ib,ie,jb,je in locate_triangle using ilo,ihi,jlo,jhi * Modified areafac for TL1, BL2, TR1 and BR2 for area flux consistency * Cosmetic changes * Added comment to explain latest change * Modification of bugcheck condition for l_fixed_area=T * update areafac_c, areafac_ce in halo in dynamics --------- Co-authored-by: apcraig <anthony.p.craig@gmail.com>
…um#875) * Added code for transport in one grid cell wide channels * Update remap advection to support transport in single gridcell channels Add single grid east and north channel configurations and tests * Update documentation * Remove temporary code comments --------- Co-authored-by: Jean-Francois Lemieux <jean-francois.lemieux@canada.ca>
* Update CICE to run with eclare108213/Icepack branch snicar (CICE-Consortium#100) * Update CICE to run with eclare108213/Icepack branch snicar - including eclare108213/Icepack#13, Sept 11, 2022 - Passes full CICE test suite on cheyenne with 3 compilers except alt04 changes answers for all compilers and all tests. CICE #fea412a55f was baseline. - Icepack submodule still points to standard version on main, need to be swapped manually to appropriate development version. * Remove faero_optics * update ciceexe string to account for USE_SNICARHC CPP * Update documentation * Update test suite to add modal testing * Point Icepack submodule to cice-consortium/E3SM-icepack-initial-integration Update to snicar branch merge, #8aef3f785ce * Add E3SM namelists for CICE. (CICE-Consortium#101) * New e3sm and e3smbgc namelist options * Update E3SM test options * Add a simple e3sm test suite * atmbndy is not actually different * Additional changes * add Tliquidus_max namelist parameter to CICE * Add Tf argument to icepack interfaces * Add constant option for tfrz_option * Fix some diagnostic prints and add to additional drivers * Update messages and change option in alt01 * Update implementation for latest version of Icepack - Update tfrz_option, add _old options for backwards bit-for-bit - Fix unittests - Add hi_min to namelist and tests * Update Icepack * Update to E3SM-Project/Icepack/cice-consortium/E3SM-icepack-initial-integration including Icepack1.3.3 release, Dec 15, 2022. * Update Icepack to E3SM-Project/Icepack #87db73ba6d93747a9, current head of cice-consortium/E3SM-icepack-initial-integration Feb 3, 2023 * Update boxchan1e and boxchan1n tests to tfrz_option = 'mushy_old' to recover Consortium main results Update Icepack to the latest hash on E3SM-Project Icepack cice-consortium/E3SM-icepack-initial-integration, #96f2fc707fc743d7 Prior commit was a merge from CICE Consortium Main, #d466031001cf447bcd64220c842dcd2707f61e9, Sept 29, 2023 * remove icepack * update icepack --------- Co-authored-by: David A. Bailey <dbailey@ucar.edu> Co-authored-by: Elizabeth Hunke <eclare@lanl.gov>
…is (CICE-Consortium#883) changes answers for some test cases, as expected. Update tfrz_option implementation to not allow _old options.
…-Consortium#886) Update Icepack to version #0c548120ce44382 Oct 16, 2023 includes NO_SNICARHC
Update documentation
* ice_flux: zero-initialize divu and shear in init_history_dyn
'divu' and 'shear' are accessed in 'accum_hist' when writing the initial
condition before they are initialized at the start of {eap, evp,
implicit_solver}. This leads to runtime error when compiling with NaN
initialization.
Zero-initialize 'divu' and 'shear' in init_history_dyn, where the
related variable 'strength' is already zero-initialized.
* ice_history_shared: disallow 'x' in history frequency variables f_*'
In the current code, nothing prevents users from leaving 'x' along with
active frequencies in the individual namelist history frequency
variables, for example:
f_aice = 'xmd'
This configuration does not work correctly, however. The corresponding
history fields are correctly defined in
ice_history_shared::define_hist_field, but since the calls to
ice_history_shared::accum_hist_field in ice_history::accum_hist are only
done after checking that the first element of each frequency variable is
not 'x', the corresponding variables in the history files are all zero.
Prevent that behaviour by actually disallowing 'x' in history frequency
variables if any other frequencies are active. To implement that, add a
check in the loop in define_hist_field, which loops through vhistfreq,
(corresponding to f_aice, etc. in ice_history). Since this subroutine
initializes 'id(:)' to zero and then writes a (non-zero) index in 'id'
for any active frequency, it suffices to check that all previous indices
are non-zero.
* ice_history: remove uneeded conditions around CMIP history variables
In ice_history::accum_hist, after the calls to accum_hist, we loop on
the different output streams, and on the history variables in the
avail_hist_fields array, to mask out land points and convert units for
each output variable.
Since 3c99e10 (Update CICE with CMIP changes. (CICE-Consortium#191), 2018-09-27), we
also use this loop to do a special treatment for some CMIP variables
(namely, averaging them only for time steps where ice is present, and
masking points where ice is absent).
This adjustment is done if the corresponding output frequency variable
(f_sithick, etc.) does not have 'x' as its first element, and if the
corresponding index in avail_hist_field for that variable/frequency
(n_sithick(ns)) is not zero. Both conditions are in fact uneeded since
they are always true.
The first condition is always true because if the variable is found in
the avail_hist_field array, which is ensured by the condition on line
3645, then necessarily its corresponding namelist output frequency won't
have 'x' as its first character (since this is enforced in
ice_history_shared::define_hist_field).
The second condition is always true because if the variable is found in
the avail_hist_field array, then necessarily its index in that array,
n_<var>(ns), is non-zero (see ice_history_shared::define_hist_field).
Remove these uneeded conditions. This commit is best viewed with
git show --color-moved --color-moved-ws=allow-indentation-change
* ice_history: use loop index directly for CMIP variables
In ice_history::accum_hist, there is a special treatment for some CMIP
variables where they are averaged only for time steps where ice is
present, and points where there is no ice are masked. This is done on
the loop on output streams (with loop index n).
This special averaging is done by accessing a2D and a3Dc using the
variable n_<var>(ns), which corresponds to the index in the
avail_hist_field array where this history variable/frequency is defined.
By construction, this index correponds to the loop index 'n', for both
the 2D and the 3D loops. Simplify the code by using 'n' directly.
* ice_history_shared: add two logical components to ice_hist_field
At the end of ice_history::accum_hist, we do a special processing for
some CMIP variables: we average them only for time steps where ice is
present, and also mask ice-free points. The code to do that is repeated
for each variable to which it applies.
In order to reduce code duplication, let's introduce two new logical
components to our 'ice_hist_field' type, defaulting them to .false., and make them optional
arguments in ice_history_shared::define_hist_field. This allows us to
avoid defining them for each output variable. We'll set them for CMIP
variables in a following commit.
* ice_history: set avg_ice_present, mask_ice_free_points for relevant CMIP variables
In the previous commit, we added two components to type ice_hist_field
(avg_ice_present and mask_ice_free_points), relating to some special
treatment for CMIP variables (whether to average only for time steps
where the ice is present and to mask ice-free points).
Set these to .true. in the call to 'define_hist_field' for the relevant
2D variables [1], and set only 'avg_ice_present' to .true. for the 3D
variables siitdthick and siitdsnthick, corresponding to the code under
the "Mask out land points and convert units" loop in
ice_history::accum_hist.
[1]
sithick
siage
sisnthick
sitemptop
sitempsnic
sitempbot
siu
siv
sidmasstranx
sistrxdtop
sistrydtop
sistrxubot
sistryubot
sicompstren
sispeed
sidir
sialb
sihc
siflswdtop
siflswutop
siflswdbot
sifllwdtop
sifllwutop
siflsenstop
siflsensupbot
sifllatstop
siflcondtop
siflcondbot
sipr
sifb
siflsaltbot
siflfwbot
siflfwdrain
sidragtop
sirdgthick
siforcetiltx
siforcetilty
siforcecoriolx
siforcecorioly
siforceintstrx
siforceintstry
* ice_history: use avg_ice_present, mask_ice_free_points to reduce duplication
Some CMIP variables are processed differently in
ice_history::accum_hist: they are averaged only for time steps when ice
is present, and points where ice is absent are masked. This processing
is repeated for each of these variables in the 2D and 3Dc loops.
To reduce code duplication, use the new components avg_ice_present and
mask_ice_free_points of ice_hist_field to perform this processing only
for variables that were defined accordingly. The relevant variables
already have those components defined as of the previous commit.
Note that we still need a separate loop for the variable 'sialb' (sea
ice albedo) to mask points below the horizon.
* Update Icepack to #f6ff8f7c4d4cb6f Split the developer guide infrastructure section from the dynamics documentation Add a coding standard section to the documentation Add a couple sentences about the state of the parameter nghost to the documentation Update opticep to use the latest main code for the unit test * update documentation
…ortium#912) * Add capability for h extension * Update documentation for hist_str * Change hist_str to hist_suffix * Change in default namelist * Update doc/source/cice_index.rst Co-authored-by: Philippe Blain <levraiphilippeblain@gmail.com> * One more hist_str --------- Co-authored-by: Philippe Blain <levraiphilippeblain@gmail.com>
…-Consortium#922) * update ghactions testing * refactor min/max global reductions, code away from huge which was giving MPI some problems.
* removal of unused variables. * moved xav to transport. Could remove commented code. Could remove xav and yav as they are zero * Move derived parameters and only allocate if needed * bugfixes for cxp, cyp... * fix index and remove commented code in ice_grid * new version of transport_remap. xav, yav array where needed. xxav, yyav parameter * Removed comments rom ice_transport_remap and arrays for nonuniform grids
Co-authored-by: John Truesdale <jet@ucar.edu>
* Added new variable vort for vorticity output * Added calc of diag vorticity for evp, vp and eap for B, C and CD grids * updated doc and ice_in file for new vorticity variable * Changed output frequency of vorticity from m to x * Added f_vort to set_nml.histall and set_nml.histdbg * Specified location of divu, shear and vort in ice_history.F90 --------- Co-authored-by: Tony Craig <apcraig@users.noreply.github.com>
Update pio and netcdf error checks --------- Co-authored-by: anton-climate <anton.steketee@anu.edu.au> Co-authored-by: Anton Steketee <79179784+anton-seaice@users.noreply.github.com>
…nsortium#931) Update PULL_REQUEST_TEMPLATE to request detailed information about changes associated with the PR. This will be useful for the commit log when squash merging the PR. --------- Co-authored-by: Philippe Blain <levraiphilippeblain@gmail.com>
…nsortium#934) In the "End-To-End Testing Procedure" section of the user guide, we instruct users to run a base suite and a test suite, but only mention checking the results of the test suite. Also mention checking the results of the base suite first, to make sure everything passes before checking the test suite. Suggested-by: Jean-Francois Lemieux <jean-francois.lemieux@canada.ca>
This provides new features for CICE IO both thru netCDF and PIO. New namelist are added to control history and restart format, hdf5 compression and chunking, the PIO rearranger, and PIO IO task control. Separate controls are provided for history and restart files. The namelist changes are for _history_format, restart_format history_rearranger, restart_rearranger history_iotasks, history_root, history_stride, restart_iotasks, restart_root, and restart_stride history_chunksize, history_deflate, restart_chunksize, restart_deflate._ In particular, - Update restart_format and history_format options to 'cdf1', 'cdf2', 'cdf5', 'hdf5', 'pnetcdf1', 'pnetcdf2', 'pnetcdf5', 'default'. The old options, 'default', 'pio_netcdf', and 'pio_pnetcdf' are still supported and backwards compatible with lcdf64, but are deprecated and no longer documented. The old options and old namelist lcdf64 are covered by the new options. Support of the old options should be removed in the future. Note that some problems were discovered when opening files with hdf5 format but reading non-hdf5 files with a spack built PIO/netCDF. As a result, the format specified for the restart read is always 'cdf1' which provides flexibility and robustness across software installs, although it may result in serial reads of hdf5 files when a parallel read could be done. - Deprecate lcdf64 namelist. This namelist is no longer needed and is covered by the new restart_format and history_format options. The namelist still exists and is backwards compatible with the old 'default', 'pio_netcdf', and 'pio_pnetcdf' format options, but is no longer documented. This should be removed in the future. - Add new namelist to control PIO pe/task setup (iotasks, root, stride) for history and restart. These settings control the PIO IO tasks. The root, stride, and iotasks are consistent with the MPI communicator. root=0 is the first MPI task. These control PIO IO performance and are usually a function of things like the IO and node hardware. See PIO for more information. CICE computes PIO iotask, root, and stride defaults for cases where -99 is passed in for some or all of these namelist. Those defaults are somewhat constrained by a bug in PIO, NCAR/ParallelIO#1986. The current implementation avoids the bug by limiting the iotasks for some MPI task counts. This is noted in ice_pio.F90. - Add new namelist to control PIO rearranger (rearranger) for history and restart. Supports 'box', 'subset', and 'default'. These control how PIO rearrangment is carried out. default is equivalent to box and the box generally performs better. See PIO for more information. - Add new namelist to support hdf5 compression and chunking (deflate, chunksize) for history and restart. The deflate controls file compression and is an integer between 0 and 9 where 0 means no compression and 9 is maximum compression. Generally, the higher the number, the slower the IO and the smaller the file, but the optimal setting depends on the contents of the file. Chunksize provides a performance control for the hdf5 parallel writes. It is a 2d array and is associated with the size of the piece of the array written by hdf5. hdf5 can be read and written in parallel, but that depends on how netCDF and PIO are built. Note that prior version of PIO, including PIO1, do not support the hdf5 compression and chunking thru the PIO interface. - Add new namelist settings (set_nml files) and update the io_suite to cover the new IO options. Remove old namelist settings associated with the deprecated format options and the lcdf64 namelist. These deprecated feature are no longer tested. - Update documentation to add new namelist and IO features. - Update the nuopc/cmeps driver code to support the new features. - Update the default ice_in to add the new namelist. - Update the derecho netcdf module to a version that supports hdf5. - Clean up some code formatting (indentation) --------- Co-authored-by: Anton Steketee <anton.steketee@anu.edu.au>
14 tasks
NickSzapiro-NOAA
approved these changes
Apr 6, 2024
* this fixes a block of code w/in a CESMCOUPLED ifdef block
DeniseWorthen
force-pushed
the
feature/pio_options
branch
from
e7d2527
to
f365592
Compare
BrianCurtis-NOAA
approved these changes
Apr 10, 2024
|
Testing for #2145 is complete, please continue with the merge process, thank you |
|
Please also verify review questions should be marked as resolved / answered. |
DeniseWorthen
added a commit
to DeniseWorthen/CICE
that referenced
this pull request
May 10, 2024
DeniseWorthen
added a commit
to DeniseWorthen/CICE
that referenced
this pull request
May 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates to latest Consortium/main.