All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
-
Implemented update in transactions. Previous only appends were handled. This allows actions which modify existing emails to be handled inside transactions when using the 'delayed_metadata' strategy.
-
Update flags in a transaction, speeding up flag refreshes by avoiding repeated writes of the JSON metadata file. This is a change inspired by this pull request: #200.
-
Deprecated Ruby 2.x Support
- Fixed a bug in single backup where the
--mirrorparameter was ignored.
- Handling of
--delimiterand--prefixoptions forimap-backup restore.
- Suppressed mailbox and metadata '...is invalid' messages when these files are missing. Now, the messages are printed when a file is actually malformed.
- Bugfix: corrected the behaviour of the legacy metadata convertor. Previous, mailboxes and metadata files would get deleted after migration.
- Full developer documentation coverage
- Bugfix: check for the presence of the spec directory before conditionally activating code coverage
- Documentation is now split into two areas:
- End-user facing: via the README on GitHub
- Developer: via a different main YARD page on rubydoc.info
- Always check whether the configuration file has folders listed as an Array of Hashes, and convert to the newer, simpler, structure. See #184
- Each release is accompanied by the creation of a container image (usable with Docker or Podman) the can be used to run backups without any other installation necessary.
- Fixed bug in account setup where changing the list of folders to back up did not set the account modified flag.
- Fixed: When handling serialization errors, when using the
delayed_metadatastrategy, previously downloaded messages are now saved.
- Messages that cause errors during serialization are now skipped end the account backup continues.
- A new
single backupcommand that backs up an account without the need for a configuration file.
- Split verbosity into two levels. The first gives local verbose logging, the second shows communications with the IMAP server.
- Fixed a bug where under certain conditions
imap-backup versionwould throw an error.
- Fixed a bug in mirror mode where new emails were appended to the .mbox file, but not to the metadata file.
- Fixed a bug where the setup connection tester was not actually attempting a connection;
- Fixed a bug where unseen messages were being marked as seen on iCloud.
- Deprecated support of Ruby 2.6.x
- An
--automatic-namespacesoption to automatically select source and destination prefixes and delimiters for themigrateandmirrorcommands.
- Fixed a bug in serialization where message lengths were calculated in Unicode codepoints, not in bytes;
- Delayed login of the IMAP client, speeding up operations where the client login was not actually necessary.
- Changed handling of messages with encoding errors in the body and the From header.
- Fixed error handling behaviour in transactions. Closes #164
- Default downloading strategy now delays writing metadata until the whole folder has been downloaded. For very large mailboxes (e.g. with >100k messages) this results in an approximately 30x speedup in download times. The strategy can be chosen in the setup program, under 'global options';
- When any account backup fails, the program now exits with a non-zero exit code.
- Improved logging output for Thunderbird exports,
- Stopped exports when the destination profile is not yet ready.
- Improved performance of large backups by avoiding unnecessary serialized IMAP data migration and validation checks.
- Previously, when help was requested on a subcommand method, the output was of listing all subcommand methods. Now, the output is now the help for that specific method.
- Handle invalid file name characters (specifically ':') in folder names. Invalid names are encoded to hex codes. As the encoding uses '%' and ';' to delimit encoded hex values, theses characters themselves are now considered 'invalid' for serialized folder names and are encoded.
- BUGFIX: Correctly accept logging options when running
migrateandmirrorcommands.
- When a single account throws an error, capture log it and keep running, so other accounts get backed up.
- BUGFIX: When an account has
folder_blacklistset but no list of configured folders, it now backs up all folders.
- Made
--resetoption onmigrateoptional. Now, existing emails in destination folders are kept.
- Options for setting source and destination delimiters for the
migrateandmirrorcommands.
- Filtered out non-standard flags from
migrateandrestore.
- An account option 'folder_blacklist'. When set the user chooses which folders to exclude from backups. Otherwise, the folders chosen by the user are the ones to include,
- Provision of JSON output for the 'remote folders' command,
- A 'remote namespaces' command to help with configuration of the 'migrate' and 'mirror' commands.
- Removed 'Experimental' warning from 'migrate' command,
- Removed 'Experimental' warning from 'export-to-thunderbird' command,
- Renamed Folder#*_flags methods,
- Improved setup account menu ordering,
- An account's connection_options can now be cleared by entering an empty string.
- Deprecated
folderscommand, replaced byremote folders.
- --config parameter to allow for non-default placing of the configuration file
- Deprecated
statuscommand, replaced bystats.
- Changed logging behaviour:
- Made info the normal logger level,
- Removed configuration 'debug' setting,
- Added a --verbose flag,
- Add a --quiet flag.
- Corrected handling of account connection options after changes.
- Added a 'mirror mode' to account configuration that changes backup behaviour:
- removes local folders that are no longer to be backed up,
- removes emails that are no longer present on the server,
- updates flags on the local backup to match those on the server.
- Added a 'mirror' command that takes a 'mirror mode' backup and copies it to another server.
- Added a '--refresh' option to the backup command that, updates flags on the local backup to match those on the server, even for accounts that are not in 'mirror mode'.
- Added backup and restore of IMAP flags (e.g. "Seen", "Draft").
- Introduced a new metadata format.
- Included data about message offsets and lengths in the new metadata to facilitate for future integrity checks.
- Added a migrator to transform the old (version 2) metadata files into the newer (version 3) files.
- Added handling for folder names supplied by the IMAP server with badly encoded names (e.g. UTF-8 instead of UTF-7)
- Improved the speed of some operations by tracking mailbox selection to avoid repeated calls to select the same mailbox.
- Added handling of append errors during migration.
- Added a workaround option for providers that set the '\Seen' flag when emails are fetched.
- Memoized connections to reduce unnecessary reconnections and the risk of exceeding server connection limits.
- Deprecated the 'status' command, in favour of the new 'stats' command.
- Added new 'stats' command, with optional JSON output.
- Resolved long running CI problem with feature specs failing due to too many active IMAP connections.
- Substituted the use of the Environment variable DOWNLOAD_BLOCK_SIZE
with an account-level setting:
multi_fetch_size.
- Refactored serialization code to simplify successive modifications to metadata serialization.
- During backup and/or restore, when there is a clash in a folder's uid_validity between the local copy and the server's copy, the local folder is now renamed using '-' as a separator (previously it was '.'). This should reduce compatibility problems with certain IMAP servers.
- The restore command now takes a single argument: the email to restore. The previous (deprecated) invocation which restores all accounts by default, or those indicated by the --accounts parameter, is still supported.
- A dedicated
migratecommand, distinct fromrestore.
- Experimental: Improved backup speed via multi fetches configurable via the Environment variable DOWNLOAD_BLOCK_SIZE. The default is still to fetch one message at a time.
- Setup menus
- add titles,
- indicate when data has been modified,
- add 'q' shortcut for quick menu exit,
- Debug output to hide passwords printed by the Net::IMAP gem.
- Configure logger to log synchronously so that output is in sync with the Mail gems debugging output.
- Updated
utils export-to-thunderbirdto work on Windows and macOS.
- Explicit permissions management on Windows. The config file inherits the permissions set on the user's home directory.
- Experimental
utils export-to-thunderbirdcommand that copies downloaded mailboxes to Thunderbird.
utils ignore-historycommand, useful for when you only want to backup future emails.
- GMail OAuth2 support. Tokens only last a few days, so this authentication method is not usable for automated backups.
localcommands to list accounts, folders and emails and to view single emails.