Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow db dumps at table level #818

Merged
merged 2 commits into from
Aug 14, 2024
Merged

Conversation

shawncrawley
Copy link
Collaborator

The context can be found in issue #817. I have updated the code that restores db dumps from s3, and modified the S3 Key regex pattern matching to look for "table" or "schema" in the appropriate place and then substitute whichever string is found into the DROP statement (e.g. DROP table IF EXISTS... or DROP schema IF EXISTS...). The proposed new format for the db dump files that would be required with this code change is:

YYYYmmddTHHMM_<db_name>DB_<"table" or "schema">_<schema[.table]>_vX_Y_Z.dump

I added the YYYYmmddTHHMM portion as well, because this solves the question of what would happen if someone dumped an entire schema after making many updates to it, but then sometime (days, weeks) afterward, another person updates a single table and then dumps just that table. The order that the dumps are applied on restore would be essential - since these act essentially like db migrations. The default already in place is that the dumps are executed in alphanumeric sorting order. Thus, with the date in the dump name, the tables would be restored in the same order they were dumped - maintaining the integrity of the database.

Though it's annoying to have to remember these dump filename formats when creating the dumps, I hope to automate this via the proposed HydroVIS Developer Tethys App - in which case it wouldn't be a big deal. But that's just a side note since the annoyance is present regardless of this update or not.

@shawncrawley shawncrawley dismissed nickchadwick-noaa’s stale review August 14, 2024 15:15

The merge-base changed after approval.

@nickchadwick-noaa nickchadwick-noaa merged commit 8f38a8f into ti Aug 14, 2024
1 check passed
@nickchadwick-noaa nickchadwick-noaa added this to the V2.1.7 milestone Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow for dumping/restoring DBs at the table level for more efficient deployments
2 participants