Import and symlink curves from ETDataset

On ETDataset all the curves (and corresponding source and data processing files) used in the ETM are stored and documented. Whenever the curves are updated, these should also be updated in the relevant ETSource country datasets by running the import_curves.rb script. If necessary, the symlinks of the local dataset curves should be updated by running the symlink_curves.rb

Country datasets

For each country dataset specified in curves.yml, the curves are imported as follows:

1. If available, use the curve for the specific country and the corresponding analysis year.
2. If the curve is available for the specific country but not for the corresponding analysis year, the curve from the nearest year is used.
3. If there is no curve available for the specific country, the (default) nl/2015 curve is used.

In order to import the curves, run rake import:curves. This runs the script for all country datasets specified in curves.yml. If the script should only be run for a specific country and year, this can be specified in the command: rake import:curves DATASET=<geo_id> YEAR=<year>.

Local datasets

For each local dataset (provinces, regions, municipalities, neighbourhoods, etc.), the curves are symlinked to the corresponding country and analysis year specified in its AREA-file. Most local datasets are 2016-based, while the nl dataset is 2015-based. Hence, a nl2016 dataset only containing the corresponding curves has been created on ETSource for the purpose of symlinking.

In order to symlink the curves, run rake symlink_curves. This runs the script for all local datasets specified in local_curves.yml. If the script should only be run for a specific dataset and year, this can be specified in the command: rake symlink_curves DATASET=<geo_id> YEAR=<year>.