-
Notifications
You must be signed in to change notification settings - Fork 15
Global Workflow
EMC_verif-global has been connected within the Global Workflow to generate statistics for a user's experiment, once the experiment is completed users can run EMC_verif-global outside of the workflow to generate graphics and display them on RZDM. A walk through presentation can be found at https://docs.google.com/presentation/d/1-XsqvaGhvJqbmhRrdCp8VZmRepa4Pedd/edit#slide=id.p1.
The scripts, configuration files, and settings below are what are and found in the global workflow (https://github.com/NOAA-EMC/global-workflow) not in EMC_verif-global
-
When a user runs sorc/checkout.sh in their global workflow copy, it will clone EMC_verif-global and check out a specific tag. It will be cloned in the sorc directory as verif-global.fd.
-
The needed climatology and verification masking files are linked through sorc/link_workflow.sh. They'll be linked in fix/fix_verif.
-
Once a user runs their set up script for the global workflow (ush/rocoto/setup_expt.py), they can set if METplus verification is to be done or not in their experiment. In config.base, the variable "DO_METP" controls whether the METplus tasks will be incorporated into the workflow XML, and thus if they will run during the experiment. The default setting is "YES". This means the METplus tasks will be incorporated into the workflow XML, and run during the experiment by default.
Once the workflow XML has been generated through ush/rocoto/setup_workflow.py or ush/rocoto/setup_workflow_fcstonly.py, the XML will show the task for the METplus verification. METplus verification is only supported for the GFS. The metatask is "gfsmetp" that has 3 tasks: gfsmetpg2g1, gfsmetpg2o1, gfsmetppcp1. gfsmetpg2g1 runs the grid-to-grid verification statistics production, gfsmetpg2o1 runs the grid-to-observations verification statistics production, and gfsmetppcp1 runs the precipitation verification statistics production. In the first GFS cycle, the metatasks will not run.
-
The configuration file for the gfsmetp tasks is parm/config/config.metp. Config.metp is set up to run "out-of-the-box" in the global workflow; a user DOES NOT need to make any edits to config.metp. However, a user may wish to make changes to various settings if they desire. The settings are described below.
The first section contains switches for the 3 cases of verification that can be run in the global workflow. The setting "YES" will run the verification, and is the default. The setting "NO" will not run the verification. If set to "NO" the corresponding metatask will still run, but will have a very short run time as the verification will not actually be run.
RUN_GRID2GRID_STEP1: grid-to-grid verification statistics production, i.e. gfsmetpg2g1 (default: YES) RUN_GRID2OBS_STEP1: grid-to-obs verification statistics production, i.e. gfsmetpg2o1 (default: YES) RUN_PRECIP_STEP1: precipitation verification statistics production, i.e. gfsmetppcp1 (default: YES)The "EMC_VERIF_GLOBAL SETTINGS" section is for settings related to the EMC_verif-global code to use when running the tasks.
HOMEverif_global: the GitHub clone or copy of EMC_verif-global to use (default: ${HOMEgfs}/sorc/verif-global.fd) VERIF_GLOBALSH: path to the mediator script between the global workflow and EMC_verif-global (default: $HOMEverif_global/ush/run_verif_global_in_global_workflow.sh)The "INPUT DATA SETTINGS" section is for settings related to the input data to be used to verify
model: name of the model or experiment (default: $PSLOT) model_dir: full path to the online directory containing the model analysis and forecast files; the code will look for the files in ${model_dir}/${model} (default: $ARCDIR/..) model_file_format: template of the model forecast file (default: pgbf{lead?fmt=%2H}.${CDUMP}.{init?fmt=%Y%m%d%H}.grib2) model_hpss_dir: full path to the directory on HPSS; the code will look for the file in ${model_hpss_dir}/${model} (default: $ATARDIR/..) get_data_from_hpss: set to "NO" to not retrieve missing model data from HPSS; set to "YES" to retrieve missing model data from HPSS, NOTE: SETTING THIS TO YES WILL POTENTIALLY INCREASE RUN TIME (default: NO) hpss_walltime: job submission wall time in minutes for jobs to get data from HPSS (default: 10)The "OUTPUT SETTINGS" section to is for settings related to the output from the metatasks
OUTPUTROOT: full path to the directory where the temporary output will be (default: $RUNDIR/$CDUMP/$CDATE/vrfy/metplus_exp) model_stat_dir: full path to the directory where the archive of METplus .stat files should be; the code will put it in ${model_stat_dir}/metplus_data (default: $ARCDIR/..) make_met_data_by: set to "VALID" to treat the verification date as a valid date; set to "INIT" to treat the verification date as an initialization date (default: VALID) SENDMETVIEWER: set to "YES" to send data to METviewer AWS; set to "NO" to not send data to METviewer AWS (default: NO) **** IMPORTANT NOTES ABOUT METVIEWER AWS **** 1. A user MUST have an account on METviewer AWS to be able to send the data. If you do not have an account and would like one please email Perry.Shafran@noaa.gov. 2. If you re-run an experiment with the same database name and for the same dates for data that has been previously been loaded into METviewer and would like the new data to be uploaded you MUST delete the database. Attempting to load data to an existing database for dates already in the database will fail as the loading XML does not allow duplicate data.The "DATE SETTINGS" section is for the setting related to the verification date to be used
VRFYBACK_HRS: number is hours to subtract from the cycle date; the resulting date will be used as the verification date (default: 24)The "METPLUS SETTINGS" section is for settings related to the logging output from MET and METplus
METplus_verbosity: verbosity of the METplus logging; options, in order from most verbose to least verbose, are: NOTSET, DEBUG, INFO, WARNING, ERROR, CRITICAL (default: INFO) MET_verbosity: verbosity of the MET logging; options are from 0-10 with 0 the least verbose and 10 the most verbose (default: 2) log_MET_output_to_METplus: set to "yes" for METplus to capture the MET output in the log; set to "no" for METplus to no capture the MET output in the log (default: yes)The "GRID-TO-GRID STEP 1" section is for settings related to the grid-to-grid verification statistics production or what is run in the gfsmetpg2g1 metatask
g2g1_type_list: list of grid-to-grid verification types to run; options: anom, pres, sfc (default: anom pres sfc) g2g1_anom_truth_name: truth to use for "anom" type; options: self_anl, self_f00, gfs_anl, gfs_f00 (default: self_anl) g2g1_anom_truth_file_format: template of the truth file for "anom" type (default: pgbanl.gfs.{valid?fmt=%Y%m%d%H}.grib2) g2g1_anom_fhr_min: forecast hour minimum to verify for "anom" type (default: $FHMIN_GFS) g2g1_anom_fhr_max: forecast hour maximum to verify for "anom" type (default: $FHMAX_GFS) g2g1_anom_grid: verification grid for "anom" type; options are valid NCEP grids (default: G002) g2g1_anom_gather_by: how to gather/organize the MET .stat files for the archive for "anom" type; options are VALID, INIT, VSDB (default: VSDB) g2g1_pres_truth_name: truth to use for "pres" type; options: self_anl, self_f00, gfs_anl, gfs_f00 (default: self_anl) g2g1_pres_truth_file_format: template of the truth file for "pres" type (default: pgbanl.gfs.{valid?fmt=%Y%m%d%H}.grib2) g2g1_pres_fhr_min: forecast hour minimum to verify for "pres" type (default: FHMIN_GFS) g2g1_pres_fhr_max: forecast hour maximum to verify for "pres" type (default: FHMAX_GFS) g2g1_pres_grid: verification grid for "pres" type; options are valid NCEP grids (default: G002) g2g1_pres_gather_by: how to gather/organize the MET .stat files for the archive for "pres" type; options are VALID, INIT, VSDB (default: VSDB) g2g1_sfc_truth_name: truth to use for "sfc" type (default: self_f00) g2g1_sfc_truth_file_format: template of the truth file for "sfc" type (default: pgbf00.gfs.{valid?fmt=%Y%m%d%H}.grib2) g2g1_sfc_fhr_min: forecast hour minimum to verify for "sfc" type (default: FHMIN_GFS) g2g1_sfc_fhr_max: forecast hour maximum to verify for "sfc" type (default: FHMAX_GFS) g2g1_sfc_grid: verification grid for "sfc" type; options are valid NCEP grids (default: G002) g2g1_sfc_gather_by: how to gather/organize the MET .stat files for the archive for "sfc" type; options are VALID, INIT, VSDB (default: VSDB) g2g1_mv_database_name: database name for the grid-to-grid data (default: mv_${PSLOT}_grid2grid_metplus) g2g1_mv_database_group: database group for the grid-to-grid database (default: NOAA NCEP) g2g1_mv_database_desc: description about the database (default: Grid-to-grid METplus data for global workflow experiment ${PSLOT})
The "GRID-TO-OBS STEP 1" section is for settings related to the grid-to-observation verification statistics production or what is run in the gfsmetpg2o1 metatask
g2o1_type_list: list of grid-to-obs verification types to run; options: upper_air, conus_sfc, polar_sfc (default: upper_air conus_sfc)
g2o1_upper_air_msg_type_list: prepbufr message type for "upper_air" type (default: ADPUPA)
g2o1_upper_air_vhr_list: list of two digit valid hours for "upper_air" type (default: 00 06 12 18)
g2o1_upper_air_fhr_min: forecast hour minimum to verify for "upper_air" type (default: $FHMIN_GFS)
g2o1_upper_air_fhr_max: forecast hour maximum to verify for "upper_air" type (default: 240)
g2o1_upper_air_grid: verification grid for "upper_air" type; options are valid NCEP grids (default: G003)
g2o1_upper_air_gather_by: how to gather/organize the MET .stat files for the archive for "upper_air" type; options are VALID, INIT, VSDB (default: VSDB)
g2o1_conus_sfc_msg_type_list: prepbufr message type for "conus_sfc" type (default: ONLYSF ADPUPA)
g2o1_conus_sfc_vhr_list: list of two digit valid hours for "conus_sfc" type (default: 00 03 06 09 12 15 18 21)
g2o1_conus_sfc_fhr_min: forecast hour minimum to verify for "conus_sfc" type (default: $FHMIN_GFS)
g2o1_conus_sfc_fhr_max: forecast hour maximum to verify for "conus_sfc" type (default: 240)
g2o1_conus_sfc_grid: verification grid for "conus_sfc" type; options are valid NCEP grids (default: G104)
g2o1_conus_sfc_gather_by: how to gather/organize the MET .stat files for the archive for "conus_sfc" type; options are VALID, INIT, VSDB (default: VSDB)
g2o1_polar_sfc_msg_type_list: message type to use from IABP data; options: IABP (default: IABP)
g2o1_polar_sfc_vhr_list: list of two digit valid hours for "polar_sfc" type (default: 00 03 06 09 12 15 18 21)
g2o1_polar_sfc_fhr_min: forecast hour minimum to verify for "polar_sfc" type (default: $FHMIN_GFS)
g2o1_polar_sfc_fhr_max: forecast hour maximum to verify for "polar_sfc" type (default: 240)
g2o1_polar_sfc_grid: verification grid for "polar_sfc" type; options are valid NCEP grids (default: G219)
g2o1_polar_sfc_gather_by: how to gather/organize the MET .stat files for the archive for "polar_sfc" type; options are VALID, INIT, VSDB (default: VSDB)
g2o1_prepbufr_data_run_hpss: set to "YES" to retrieve missing prepbufr files from HPSS; set to "NO" to not retrieve missing from HPSS (default: NO)
g2o1_mv_database_name: database name for the grid-to-obs data (default: mv_${PSLOT}_grid2obs_metplus)
g2o1_mv_database_group: database group for the grid-to-obs database (default: NOAA NCEP)
g2o1_mv_database_desc: description about the database (default: Grid-to-obs METplus data for global workflow experiment ${PSLOT})
The "PRECIP STEP 1" section is for settings related to the precipitation verification statistics production or what is run in the gfsmetppcp1 metatask
precip1_type_list: list of precipitation verification types to run; options: ccpa_accum24hr (default: ccpa_accum24hr)
precip1_ccpa_accum24hr_model_bucket: two digit model accumulation bucket or "continuous" to use the accumulation over the model run (default: 06)
precip1_ccpa_accum24hr_model_var: model precipitation variable to use; options: APCP, PRATE (default: APCP)
precip1_ccpa_accum24hr_model_file_format: template of the model file for "ccpa_accum24hr" type; this overrides model_file_format (default: pgbf{lead?fmt=%2H}.gfs.{init?fmt=%Y%m%d%H}.grib2)
precip1_ccpa_accum24hr_fhr_min: forecast hour minimum to verify for "ccpa_accum24hr" type (default: $FHMIN_GFS)
precip1_ccpa_accum24hr_fhr_max: forecast hour maximum to verify for "ccpa_accum24hr" type (default: 180)
precip1_ccpa_accum24hr_grid: verification grid for "ccpa_accum24hr" type; options are valid NCEP grids (default: G211)
precip1_ccpa_accum24hr_gather_by: how to gather/organize the MET .stat files for the archive for "ccpa_accum24hr" type; options are VALID, INIT, VSDB (default: VSDB)
precip1_obs_data_run_hpss: set to "YES" to retrieve missing observation files from HPSS; set to "NO" to not retrieve missing from HPSS (default: NO)
precip1_mv_database_name: database name for the precipitation data (default: mv_${PSLOT}_precip_metplus)
precip1_mv_database_group: database group for the precipitation database (default: NOAA NCEP)
precip1_mv_database_desc: description about the database (default: Precip METplus data for global workflow experiment ${PSLOT})
- There will be two locations for your output. The first is the temporary location of the running of the "gfsmetp" tasks. That will be found in
$OUTPUTROOT. The second is the permanent location of the archive of the MET stat files. That location is found in $ {model_stat_dir}/metplus_data. There are various sub-directories beneath that such as the "gather_by" method, verification case, verification type, initialization/valid hour, and model name. In the model directory there should be daily verification files.
Once the experiment is complete, a user will want to plot the statistical output to see where there are improvements or degradations compared to the operational GFS or GFS version 16. For this a user will need to run EMC_verif-global in a standalone way.
Since the code was already checked out within the global workflow under sorc/verif-global.fd, users can use this copy. Users are also welcome to again checkout EMC_verif-global in another location and use that copy.
Regardless of location of the EMC_verif-global being used. Users will need to set up a configuration file and run the code. Please see the standalone section on how to run and set up the configuration file at https://github.com/NOAA-EMC/EMC_verif-global/wiki/Standalone.
The generation of fit-to-obs graphics has not been incorporated into EMC_verif-global. Users can generate their graphics using the fit-to-obs plotting package. Once the graphics have been created and sent to RZDM, users can the script fit2obs/link_fit2obs.py that is located on in their web directory on RZDM. The script takes 3 runtime arguments: 1) path to Fit-to-Obs graphics, 2) experiment 1 name, and 3) experiment 2 name. The script will link to the fit-to-obs graphics for display on the website. Below is an example of running the script
python link_fit2obs.py /home/www/emc/htdocs/gmb/emc.glopara/vsdb/v161rt1/fits fnl v161rt1