Add sync summaries and deletion notifications

[gmaclennan]

Sync can be confusing for users. If another user has changed a lot of data, in particular deleted data, then syncing can result in a users dataset suddenly changing and data disappearing (because deletions were synced). We have seen that this can cause users to think that data is lost because of an error with Mapeo or a problem with sync.

It would be good to show the user details about what changed due to a recent sync, perhaps a summary with the option to see details e.g.

Sync with XX device on MMM,DD,YYYY

• N new observations from M users
• Y observations were changed by S users
• Z observations were deleted by T users

Then clicking on each of the summary points could show a list of all the points edited/created/deleted and the user can click on them to see the actual point, with the option to undelete or undo an edit, or it could show an intermediate summary grouped by user first.

If a sync results in a lot of data being deleted, we could show some kind of warning / notification, with a clear pointer to the user about how they can see and recover this data. E.g. a user syncs with someone who has deleted a bunch of points by mistake, and they sync and suddenly they see data disappear, we can show a notification like "The last sync included XX deleted points from user YYY. You can see the data that was deleted by clicking here and you can undo the deletions if this was a mistake".

The summary could be very reassuring for users after sync, so that they can see what has actually happened, and it could help users understand a bit more what is going on when they sync.

Questions

@jencastrodoesstuff @aliya-ryan @aldopuicon

1. What is the minimal viable implementation of this?
2. Is a warning about deletions or a summary of changes a bigger priority?
3. How many deletions should trigger a warning?
4. How can we show edit history and provide an easy way to undo changes?
5. What should this look like on Desktop and mobile
6. Should we prioritise Desktop or Mobile first?

Technical implementation

@okdistribute @noffle

I think to implement this we need to record sync events to the DB, with a current state of the hypercores recorded in the sync event. If we want a history of not just the most recent sync, but all sync events, then we might need to record the state at both start sync and end sync. We can then derive the changes from that. We would need to add a method to mapeo-core to read this data.

For this to be more useful we probably need at a minimum device names, so that we can track down which device was responsible. It would also be useful to have a way that users can record their name when using the app (as we have discussed previously) so that we can also assign edits/deletions to a person. This should be optional so that we can support anonymous usage where it is appropriate.

[hackergrrl]

I think to implement this we need to record sync events to the DB, with a current state of the hypercores recorded in the sync event. If we want a history of not just the most recent sync, but all sync events, then we might need to record the state at both _start sync_ and _end sync_. We can then derive the changes from that. We would need to add a method to mapeo-core to read this data.

I think we'd only need sync-end records, since sync-start can be inferred by comparing the previous sync-end (minus whatever local entries you wrote). This should be enough to derive any prior state. Storing the full state for every feed per sync record will add up quickly, but if we stored just the deltas (feeds that we synced new data from this time) it would reduce the record size dramatically, since most feeds don't change between syncs. From this, it wouldn't be hard to write a materialized view that provided sync state information across every device's history.

For this to be more useful we probably need at a minimum device names

This seems like a pretty easy improvement. You could then, at the very least, as a user, see who did the deletions, and go talk to them.

[okdistribute]

Yes, and device names have been a long-awaited feature request.