RELEASE_NOTES.txt

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Land Verification Toolkit (LVT) Release Notes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


.. admonition:: **Current release:**

          Land Verification Toolkit (LVT) Public Release 7.2r

          6 May 2015


===========
CHANGE LOG:
===========


================================================================
Land Verification Toolkit (LVT) Public Release 7.2r (r5927:7374)
================================================================

This is the public release of LVT 7.2.

These updates consist of:

- Corrected error message to logfile when the option
  for NLDAS2 hourly or monthly is not in the config.

- Added the ability to use the NLDAS-2 monthly datasets
  instead of only the NLDAS-2 hourly datasets.  This is
  only appropriate for when using LVT to calculate on a
  monthly basis.  When using sub-monthly (daily/hourly),
  then the NLDAS-2 hourly datasets should be used.

  Note that this code will only work with NLDAS-2 Noah,
  Mosaic, and VIC for the monthly datasets.  There is
  no NLDAS-2 SAC dataset currently available.  The code
  continues to support NLDAS-2 hourly datasets for all
  four LSMs, including SAC.

  Also fixed the RootMoist top 1-m soil moisture field
  for NLDAS-2 Mosaic and VIC (for both the hourly and
  monthly datasets).

- Fix for when calculating the RootMoist variable
  from the SoilMoist LIS output.  This fix applies
  when the RootMoist variable was not saved in the
  LIS run.

- Added a fix to enable rootzone soil moisture calculations in the NLDAS2
  reader without explicitly enabling the individual snow layers

- restricting the use of USCRN data to the quality controlled stations only
  SRI fix to allow calculations based on either total runoff or streamflow

- Added two plugins

  1. To read the US drought monitor gridded data

  2. To read the LVT generated percentiles as an observation source

- Adding the missing gaussian_routines.F90 file

- bug fix: in certain metric computations, negative values were 
  being excluded

- Fixed a bug in the variance and standard deviation calculations when 
  both datastreams are used simultaneously. 

  Updated the DA diagnostics reader to process analysis increments

- Added plugins for the following datasets

  1.FLUXNET2015 - synthesized station FLUXNET data

  2.GLEAM - evaporation data from GLEAM

- Added the support for the zscore metric (value - mean)/standard.deviation

- Updates to enable the choice of a particular level when working with a
  3d field

  The specification of 'LVT datastream attributes table::' has changed. 

  The older format is as follows, where the first layer from a 
  soil moisture field with 4 levels (datastream 1) is compared against 
  first level from the soil moisture field with one level (datastream 2)

  ::

    LVT datastream attributes table::
    SoilMoist  1  m3/m3   - 1 4 SoilMoist  1 m3/m3    - 1 1
    ::

  The equivalent new specification is as follows::

    LVT datastream attributes table::
    SoilMoist  1 1  m3/m3   - 1 4 SoilMoist  1 1 m3/m3    - 1 1
    ::

  Note that there are two values (column 2 and 3, 9 and 10) that
  specify the starting and ending indexes of the levels to be used. 

  If the user wants to use the 3rd soil moisture level in the
  comparisons, the specification will be::

    LVT datastream attributes table::
    SoilMoist  3 3  m3/m3   - 1 4 SoilMoist  1 1 m3/m3    - 1 1
    ::

  Note that the older selection did not allow the specific 
  selection of a particular level.

- Updated the GIMMS AVHRR NDVI reader; fixed some of the
  filename paths as the years/AVHRR-satellite numbers were
  incorrect.

- Added a plugin to process the USCRN soil moisture and soil temperature data

- Fix to CMORPH precipitation units.

  The CPC CMORPH GrADS control file lists the units as mm/3hr, but comparisons
  with the daily CPC Unified Gauge-Based Analysis suggest the CMORPH values
  are really 3-hr accumulations instead of rates.  The correction removes
  the multiplication of 3.  (Before this correction, the 3-month sums
  of CMORPH values was about 3 times larger than the CPC analysis, domain-wide 
  and for most pixel-by-pixel comparisons.)

- Bug fixes to CPC-Unified data reader:

  (1) Changed code to only read data at 00Z.  This allows comparison of this
      product against sub-daily data.
  (2) Changed logic to read the prior day's file.  This is because the 
      CPC-Unified analysis is a "forward" accumulation covering 00Z to 23:59Z.
  (3) Fixed unit conversions.

- Bug fixes to CMORPH reader:

  (1) Changed code to read data from 3 hours prior, since CMORPH is a "forward"
      accumulation data stream (e.g., 00Z data is for 00Z through 02:59Z).
  (2) Added convert="little_endian" option to open statement.  While this is
      not standard Fortran, it appears to be a widely supported compiler
      extension.
  (3) Fixes to unit conversions.
  (4) Improved logic to stop reading from CMORPH file once desired time slice
      has been extracted.
  (5) Changed filename logic to read the "adjusted" (i.e., bias-corrected)
      CMORPH products.

  TODO:

  (1) Add ProTex documentation.
  (2) Allow user to read either raw or bias-adjusted CMORPH files.

- Added INFO print statements.

- added SPI timescale of computation: to config.master

- added a plugin for SMOS (NESDIS) retrievals

- First cut at supporting CMORPH precipitation data in LVT.

  The current plugin support 0.25 deg, 3-hourly CMORPH little-endian files from
  NOAA CPC.

  TODOs:

  * More robust support for little-endian files (currently requires whole 
    program to be compiled for "little-endian")
  * ProTex documentation in new files.
  * POSSIBLE:  Support for daily data instead of 3-hrly.
  * POSSIBLE:  Support for 30-min/8-km resolution data.

- Added status message indicating when NLDAS2 file cannot be found.

  The new INFO message can be useful when diagnosing run-time problems with
  LVT (such as a bad path to the NLDAS2 files).

- Added status message indicating when AGRMET file cannot be found.

  This can occur if the LVT time step is below three hours (e.g., when 
  comparing AGRMET to NLDAS2).  

  The new INFO message can be useful when diagnosing run-time problems with
  LVT (such as a bad path to the AGRMET files).

- Adding the missing plugin for simulated GRACE data

- Added a plugin for gridded USGS streamflow data 

  Updates to the DA diagnostics plugin

- Added the processing of NLDAS-2 forcing variables to LVT.

- Added a plugin to process the Daymet SWE data 

- Updated to the new format of the USGS data

- Adding a plugin for the native SCAN data 

- Moving the default SCAN plugin to a "SCANGMAO plugin" 

  A separate plugin that relies on the raw SCAN data is being added so that we are not reliant on GMAO for more recent datasets

- updated SRI and SSWI computations based on the fixes to SPI

- updated SPI plugin 

- fix for the new ARS data files

- Updates to the ARS soil moisture reader to work with the updated ARS dataset

- another bug fix to the SPI computation

- bug fix to the SPI computations for different timescales 

- Fixed a bug in the GIMMS MODIS NDVI reader that causes the processsing
  to return undefined values after the first successful read

- added the rank correlation option to the master metrics.tbl

- improved error msg, instructing user to see lvt.config.master

- changed the lvt.config_master file to specifiy GIMMS AVHRR NDVI

- Added a warning message to the log diagnostic
  writing when ESA CCI SM file is not being read,
  mainly occurs when a directory path is too long
  in the ESA CCI SM reader.

- Bug fix to make sure that the outputs from the MEAN computations are enabled
  when simultaneously used with a metric that requires multiple passes through
  the data. 

- Added support for Rank correlation computations

- Updated the MOD10A1 reader to work with the NetCDF format

- bug fix in Anomaly metric code - related to the output 

  Added the flexibility to do 3-month, 6-month, 9-month SPI, SRI, SSWI
  calculations

- some cleanups, separating the plugin for FLUXNET mte and station data,

  Added a plugin to process LIS DA diagnostics

  LISDA obs plugin updated to support interpolation to the LVT grid

- Minor tweaks to the config file options 

  Added the support for GIMMS MODIS NDVI data 

  Changed the old GIMMS NDVI reader to GIMMS AVHRR NDVI reader 

  Started the plugin for FLUXNET station data

- changed the FLUXNET to FLUXNET MTE (will be adding a separate plugin for FLUXNET station data)

  GL6JULES plugin has been rewritten as JULES data plugin

  Changes to handle more derived variables that require multiple passes

- for ESA CCI plug in added support for v2.2
  for SSEB fixed upside-down input interpretation

- Updated lvt.config.master file to reflect recent changes
  to config lines, which were changed based on user inputs
  from confusion with the meaning of the older options.

  ``Temporal averaging interval:`` changed to ``Metric computation frequency:``

  ``Stats output interval:`` changed to ``Metric output frequency:``

  ``Stats output directory:`` changed to ``Metric output directory:``

- cleaning up the updated options to be consistent:

  ``Stats output directory:`` option changed to ``Metric output directory:``

- Changing two key lvt.config options: 

  The ``Temporal averaging interval:`` option is deprecated and has been changed
  to ``Metric computation frequency:``

  The ``Stats output interval:`` has been changed to 
  ``Metric output frequency:`` 

  These changes are based on user inputs/confusion about the meaning of the
  older options

- added upscaling capabilities to the MOD10A1 reader

- adding plugins for AMSR2 snow depth and SNODEP

- Adding a plugin for supporting LIS6 outputs (currently customized
  for the operational NOHRSC outputs only)

- Update the README_discover file

- Correct length of ansa_filename

  I also cleaned up some diagnostic log messages by adding trim around
  references to ansa_filename.

- Merge NU-WRF updates into tools/lvt/7/public7.1

  I manually merged commit r3042 from the v7_lis71merge branch
  of the NU-WRF repository on progress,
  svn/nu-wrf/code/branches/features/v7_lis71merge, into
  LIS' tools/lvt/7/public7.1 branch.

  These updates consist of:

  * Bug fix: correct the order of endif/#endif statements

- Document the "WMO convention (AFW OPS)" option for "LIS output naming style:"

- Added new "LIS output naming style:" option::

    "WMO convention (AFW OPS)"

  Also added some additional lvtlog statements
  and checks for available naming styles and
  LIS output filenames.

- Update documentation regarding ``Temporal averaging interval:``

  Note that the ending time and the ``Temporal averaging interval:`` must be
  consistent.  For example, for a ``Temporal averaging interval:`` of 1da, you
  must have an ending time of::

    Ending hour: 0
    Ending minute: 0 
    Ending second: 0


================================================================
Land Verification Toolkit (LVT) Public Release 7.1r (r5471:5926)
================================================================

This is the public release of LVT 7.1.

These updates consist of:

- Incorporate Eric Kemp's Makefile update

- Update documentation

- bug fix related to the output of summary and final metric files

- Add support to choose between big and little endian for linux_ifc systems

- plugins to process the Great Lakes Hydro data

- Added plugins for MODIS LST, updates to GRACE plugin to support the 13 day
  product, Support for temproal smoothing of observations

- Added the capability to do time lagged computations

- Minor formatting clean-ups.

- Added options when processing observations.
  Can now write data in LIS-style for Routing,
  RTM, and Irrigation, in addition to LSM.

- Added new units for evaporation flux components.

- Added additional NLDAS-2 variables available to
  be processed by LVT.  Includes an overhaul and
  streamline of the code.

- Added a SMOS L1 Tb reader

- Added support for GCOMW, ASCAT and SMOS



========================================================================
Land Verification Toolkit (LVT) Public Release 7.0r (r4080:5470)
========================================================================

This patch release includes updates r4080:5464

- Merge tools/lvt/7/development into tools/lvt/7/public7.0

  These updates consist of:

  * fixes for gfortran and pointer to allocatable conversions

- Add more descriptive NetCDF prompts (thanks Shugong)

- Remove MPI from configure scripts.

- Merge tools/lvt/7/development into tools/lvt/7/public7.0

  These updates consist of:

  * documentation update

  * bug fixes to CPC precip plugin

    updated GHCN to process precip obs

    updates to the history reader to be consistent with the LIS outputs (This
    enables LVT to process ALL outputs (including Lake, RTM, SAC/VIC) from LIS).

  * added the changes to the configuration script from Eric Kemp

- Create LVT 7.0 Public branch

- Rename tools/lvt-dev to tools/lvt/7/development

- Rename tools/lvt to tools/lvt-dev

- checks to avoid diagnose errors when using binary LIS outputs

- Updates to the NLDAS-2 reader and module.

  Converted the NLDAS-2 VIC reader to the GES DISC style.
  This data is available on Discover here:
  /discover/nobackup/projects/drought/shared/NLDAS2.VIC
  The data is available at the GES DISC here:
  http://disc.sci.gsfc.nasa.gov/hydrology/data-holdings

  Added NLDAS-2 SAC datasets, including the post-processed
  SAC soil moisture on Noah levels.
  The SAC data (NCEP/EMC style) is on Discover here:
  /discover/nobackup/projects/drought/shared/NLDAS2.SAC

  The post-processed soil moisture data on Discover here:
  /discover/nobackup/projects/drought/shared/NLDAS2.SAC/SM

  The data is available at NCEP/EMC here:
  http://www.emc.ncep.noaa.gov/mmb/nldas/

  GES DISC style data for the NLDAS-2 SAC datasets are not
  yet created; once they are, this plugin will be modified.

- updated the master config file

- Added the capability to compute metrics on a basin/region scale rather
  than on a pixel-by-pixel basis. If this option is turned on, LVT expects
  the user to provide a categorical map which will then be used to
  generate the basin/region delineations

- added the capability to process

  USGS ground water well data

  Plate Boundary Observatory data (snow depth and soil moisture)

- Added the plugin to read the North American Soil Moisture Database (NASMD)
  data

- fixed a missing parameter projection initialization in LVT

- A few diagnostic checks on the LVT configuration

- updated the ALEXI plugin to support the newly obtained 4km data

- Relaxed the requirement that LIS model outputs be present at every LIS output
  interval. The user can select a LIS model output interval (in lvt.config) that
  is smaller than or equal to the LIS model output interval. LVT runs its clock
  at this interval and keeps looking for LIS output files.  

- fixed a bug that caused time series files to not work when multiple soil
  layers were enabled for looking at variables like soil moisture. 

- Modifications to support the change  in NETCDF output in LIS. Backward
  compatibility to older LIS7 outputs is also maintained. 

- bug fix to the ensemble standard deviation code

- added support to read dekadal outputs from LIS

  This implementation currently does not support temporal averaging at 
  intervals other than 'dekad'

- added the support for default options to the configure script

- Corrected the number of seconds in a year.  Because this value
  is both set and used in the code, this revision will not change
  any output.

- added support for ground water well data - not fully tested yet!

  added a check in the mean metric computations to check if observations
  are enabled. 

- Adjust LDFLAGS

  Using the new JasPer library installed on discover required reordering
  some options in the LDFLAGS variable.

- Merge NU-WRF updates into tools/lvt

  I manually merged commits r642:1752 from the trunk of the NU-WRF
  repository on progress, svn/nu-wrf/code/trunk, into LIS' tools/lvt
  branch.

  These updates consist of:

  * Modify strings within calls to register_dataEntry to appease
    the gfortran compiler.

    Note that both the IBM/AIX Fortran compiler and gfortran expect
    the individual strings defined in an array like ::

       (/"string1", "string2"/)

    to have the same length.

    For example, this is incorrect ::

       (/"kg/m2s", "kg/m2"/)

    and this is correct ::

       (/"kg/m2s", "kg/m2 "/)

  * LVT_rc%var_based_strat is an integer.  To appease the gfortran
    compiler do not check it like a logical within a conditional
    statement.

- Add JasPer library to the Config.pl configure script

  Note that the user must set an LVT_JASPER environment variable
  for the configure script to work.

- added the metric to compute percentiles for each ensemble member separately

- fixes to the restart support for percentiles and ensemble percentile
  calculations

- Added the description of the option for yearly temporal averaging
  to start at any given month into the lvt.config master file.

- Added longer possible character lengths for the
  diagnostic file and for output stats directory.

- Added option for calculating only percentiles climatology
  to the lvt.config master file, and made some formatting
  improvements.

- Corrected name of NASA AMSR-E entry in lvt.config file.

- More formatting corrections to the lvt.config master file.

- Organization and LaTeX formatting changes to the
  lvt.config master file for the LVT User's Guide.

- documentation update, small update to the grib reader 

- Adding the updated LVT usersguide to the source code

- documentation updates, updates to the NCL plotting utilities. 

- Updates to config files reflecting latest additions
  to the LVT code.

- Adding the files for GRACE plugin and Anomaly computations

- 1. Added a plugin to process GRACE observations (these are the "processed"
     GRACE observations generated by LDT
  2. Added a plugin to read and process the UW ET data
  3. Updates to the indices calculations to restart from a prescribed
     climatology
  4. Bug fix so that LVT doesn't overwrite metric calculations that require
     only one pass through the data when metric types that require two passes
     through the data are simultaneously used

- Added recent option updates made within the code to the
  sample METRICS and lvt.config.master files.  Also made
  the MODEL TBL file consistent with LIS-7's config file.
  Also updated documentation for all three files, and
  made some cosmetic changes.

- Corrected some typos of gswi to sgwi.

- added the options for restart for SRI, SPI, SSWI and percentile calculations

  Previously the code computed the climatologies and the indices all in one go.
  Now the code has an option to simply compute the climatology alone and then
  use that climatology to (in a restart mode) to find the required index eor
  any particular simulation.                                                                      
  A new metric called SGWI - standardized ground water index -- which is 
  similar to SSWI, but for TWS variable.  

- support for a number of ratio variables are added:
  1. SWE/P (snow water equivalent over precip) 
  2. ET/P  (evapotranspiration over precip)
  3. Qs/P  (surface runoff over precip)
  4. Qsb/P (subsurface runoff over precip)

  Note that the model (LIS) output should have the component variables
  in the same units. LVT do not currently do unit conversions when
  computing these ratios. 

- Change to the reader of the MOD16 dataset, so LVT will point to the
  "LE" filenames instead of the "ET" filenames.  A unit conversion is
  also performed.  This change was made so users can compare fluxes in
  W/m^2.

- added the file with the actual implementation of RFV metric

- fixed a typo in the monthly time averaging interval check

- added the river flow variate metric

- Added the ability to define the start of the year when yearly intervals
  are used in time averaging

  Added metrics for computing min, max and sum

- fix to the ARM data reader to check for array out of bounds

- Set LVT_rc%lnc(n) and LVT_rc%lnr(n) within
  utils/input_proc/readinput_polar.F90

- Set LVT_rc%lnc(n) and LVT_rc%lnr(n) within
  utils/input_proc/readinput_lambert.F90

- updates to for the fix of time averaging options

- Adding length to character strings to accomodate
  some longer path names in the lvt.config file.

  Replacing the Jan 2012 LVT User's Guide with
  the latest available version from Dec 2012.

- initial version of ALEXI reader (does not work currently)

  data plugin to read LIS DA processed observations

  bug fix for monthly/yearly averaging logic 

  allowance for subsetting of the LDT processed domain


========================================================================
Land Verification Toolkit (LVT) Public Release 1.2 (r4016:4079)
========================================================================

- updates related to OptUE processing


========================================================================
Land Verification Toolkit (LVT) Public Release 1.1 (r3868:4015)
========================================================================

- updates and cleanups related to the latest round of LIS6 testing

- bug fix to the input_proc utility that fixes a bug

- Added the plugin for processing processed LIS DA observations

- updates: 

  1. removed the FEWSNET SOS processing metric     
  2. Added plugins for NLDAS2, Natural streamflow
  3. Cleaned up the utility that generates the LDT-like input file

- Add Shugong Wang's updates
  These updates provide backwards support for LIS 6.1.


========================================================================
Land Verification Toolkit (LVT) Public Release 1.0 (r3867)
========================================================================

- Initial release