Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Safer incremental backups #244

Open
lukasbestle opened this issue Feb 5, 2024 · 2 comments
Open

Safer incremental backups #244

lukasbestle opened this issue Feb 5, 2024 · 2 comments

Comments

@lukasbestle
Copy link

lukasbestle commented Feb 5, 2024

Status quo

At the moment, incremental backups always use the last update of the backup as a since date, even if the last backup failed. This is documented as a known issue.

Proposed feature

The behavior could be improved by storing the timestamps of the last successful updates in a file, e.g. /.successful-timestamps.json:

{
    "some-repo/issues": 1707118232,
    "some-repo/pulls": 1707118232,
    // ...
}

Then failed backups would not break future incremental backups.

Migration

The implementation would need to be backwards-compatible with existing backup stores that don't have this file yet or where the file is incomplete. It could work like this:

  1. If the file exists and contains the key for the backup source in question, use the timestamp as the since time.
  2. If the file doesn't exist or does not contain the relevant key but an incremental backup was requested, use the old behavior.
  3. In any case (full or incremental backup), create or update the file with the new timestamps for the next run(s).
@josegonzalez
Copy link
Owner

This project is considered feature complete for the primary maintainer @josegonzalez. If you would like a bugfix or enhancement, pull requests are welcome. Feel free to contact the maintainer for consulting estimates if you'd like to sponsor the work instead.

@hkmaly
Copy link

hkmaly commented Jan 16, 2025

I'm not sure why bother with generating separate file. There is already information of when was any specific file updated, in form of filesystem's modification time of the file. And yes, I will try to make the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants