Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow db dumps at table level #818

Merged
merged 2 commits into from
Aug 14, 2024
Merged

Allow db dumps at table level #818

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ac3e6fc
Merge branch 'hotfix-pr-qpe-description' into uat
nickchadwick-noaa Jul 15, 2024
768e30e
Implements ability to restore a table or schema
shawncrawley Aug 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions Core/EC2/RDSBastion/scripts/utils/restore_db_dumps_from_s3.sh.tftpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ export PGPASSWORD=${db_password}
%{ for s3_key in s3_key_list ~}

# Get DB and Schema name from s3_key
# (filepath)/((DB Name)DB_(Schema Name)_vX_X_X.dump)
RE='^(.+)\/(([a-z].+)DB_(.+)_v[0-9]+_.+\.dump)$'
# (filepath)/(dump date in YYYYmmddTHHMM)_(DB Name)DB_("table" or "schema")_(Schema Name).(optional table name)_vX_X_X.dump)
RE='^(.+)\/[0-9]{8}T[0-9]{4}_(([a-z].+)DB_(schema|table)_(.+)_v[0-9]+_.+\.dump)$'
[[ ${s3_key} =~ $RE ]]

# Drop old schema and restore from S3 object
echo "Setting up $${BASH_REMATCH[4]} schema in the $${BASH_REMATCH[3]} DB..."
psql -h "${db_host}" -U "${db_username}" -p ${db_port} -d "${db_name}" -qtAc "DROP SCHEMA IF EXISTS $${BASH_REMATCH[4]} CASCADE;"
echo "Setting up $${BASH_REMATCH[5]} $${BASH_REMATCH[4]} in the $${BASH_REMATCH[3]} DB..."
psql -h "${db_host}" -U "${db_username}" -p ${db_port} -d "${db_name}" -qtAc "DROP $${BASH_REMATCH[4]} IF EXISTS $${BASH_REMATCH[5]} CASCADE;"
aws s3 cp "s3://${s3_bucket}/${s3_key}" "$${postgres_data_folder}/db.dump" --only-show-errors
pg_restore -h "${db_host}" -p ${db_port} -d "${db_name}" -U ${db_username} --clean --if-exists -j 4 "$${postgres_data_folder}/db.dump"

Expand Down
9 changes: 4 additions & 5 deletions ...ish_service/services/forcing_analysis_assim_hawaii/ana_past_72hr_accum_precip_hi_noaa.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
service: ana_past_72hr_accum_precip_hi_noaa
summary: NAM-NEST NWP Past 72-Hour Accumulated Precipitation for Hawaii
description: 'Depicts accumulated precipitation totals over the past 72 hours derived from the NAM-NEST NWP forcing for
the operational National Water Model (NWM) analysis and assimilation for the state of Hawaii. Updated hourly.'
tags: nam-nest, nwp, accumulated, precipitation, national, water, model, nwm, hawaii, hi, precip
summary: MRMS Past 72-Hour Accumulated Precipitation Analysis for Hawaii
description: 'Depicts accumulated precipitation totals over the past 72 hours derived from the MRMS forcing for the operational National Water Model (NWM) analysis and assimilation for the state of Hawaii. If MRMS forcing data is not available, forcing data from the NAM-Nest is used as a backup. Updated hourly.'
tags: mrms, accumulated, precipitation, national, water, model, nwm, hawaii, hi, precip
credits: National Water Model, NOAA/NWS National Water Center
egis_server: image
egis_folder: nwm
feature_service: false
public_service: false
public_service: false
10 changes: 4 additions & 6 deletions ...rvice/services/forcing_analysis_assim_puertorico/ana_past_72hr_accum_precip_prvi_noaa.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
service: ana_past_72hr_accum_precip_prvi_noaa
summary: NAM-NEST NWP Past 72-Hour Accumulated Precipitationn for Puerto Rico and Virgin Islands
description: 'Depicts accumulated precipitation totals over the past 72 hours derived from the NAM-NEST and HIRES WRF-ARW NWP
forcing for the operational National Water Model (NWM) analysis and assimilation for Puerto Rico and the U.S. Virgin Islands.
Updated hourly.'
tags: nam-nest, nwp, accumulated, precipitation, national, water, model, nwm, puertorico, prvi, precip
summary: MRMS Past 72-Hour Accumulated Precipitation Analysis for Puerto Rico and Virgin Islands
description: 'Depicts accumulated precipitation totals over the past 72 hours derived from the MRMS forcing for the operational National Water Model (NWM) analysis and assimilation for Puerto Rico and the U.S. Virgin Islands. If MRMS forcing data is not available, forcing data from the NAM-Nest is used as a backup. Updated hourly.'
tags: mrms, accumulated, precipitation, national, water, model, nwm, puertorico, prvi, precip
credits: National Water Model, NOAA/NWS National Water Center
egis_server: image
egis_folder: nwm
feature_service: false
public_service: false
public_service: false
10 changes: 4 additions & 6 deletions ...viz_publish_service/services/forcing_short_range_hawaii/srf_48hr_accum_precip_hi_noaa.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
service: srf_48hr_accum_precip_hi_noaa
summary: NAM-NEST NWP Short-Range Accumulated Precipitation for Hawaii
description: 'Depicts expected accumulated precipitation totals over the next 48 hours derived from the NAM-Nest with
HIRESW WRF-ARW forcing for the operational National Water Model (NWM) short-range forecast for the state of Hawaii.
Updated every 12 hours.'
tags: nam-nest, nwp, accumulated, precipitation, national, water, model, nwm, hawaii, hi, precip
summary: WRF-ARW 48-Hour Accumulated Precipitation Forecast for Hawaii
description: 'Depicts expected accumulated precipitation totals over the next 48 hours derived from the WRF-ARW forcing for the operational National Water Model (NWM) short-range forecast for the state of Hawaii. Updated every 12 hours.'
tags: wrf-arw, accumulated, precipitation, national, water, model, nwm, hawaii, hi, precip
credits: National Water Model, NOAA/NWS National Water Center
egis_server: image
egis_folder: nwm
feature_service: false
public_service: false
public_service: false
10 changes: 4 additions & 6 deletions ...blish_service/services/forcing_short_range_puertorico/srf_48hr_accum_precip_prvi_noaa.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
service: srf_48hr_accum_precip_prvi_noaa
summary: NAM-NEST NWP Short-Range Accumulated Precipitationn for Puerto Rico and Virgin Islands
description: 'Depicts expected accumulated precipitation totals over the next 48 hours derived from the NAM-Nest with
HIRESW WRF-ARW forcing for the operational National Water Model (NWM) short-range forecast for Puerto Rico and the
U.S. Virgin Islands. Updated every 12 hours.'
tags: nam-nest, nwp, accumulated, precipitation, national, water, model, nwm, puertorico, prvi, precip
summary: WRF-ARW 48-Hour Accumulated Precipitation Forecast for Puerto Rico and Virgin Islands
description: 'Depicts expected accumulated precipitation totals over the next 48 hours derived from the WRF-ARW forcing for the operational National Water Model (NWM) short-range forecast for Puerto Rico and the U.S. Virgin Islands. Updated every 12 hours.'
tags: wrf-arw, accumulated, precipitation, national, water, model, nwm, puertorico, prvi, precip
credits: National Water Model, NOAA/NWS National Water Center
egis_server: image
egis_folder: nwm
feature_service: false
public_service: false
public_service: false
Loading