v2.1.0

reproducible 2.1.0

New

• new family of functions that are called inside postProcessTo that use sf::gdal_utils directly. These are still experimental and will only be activated with options("reproducible.gdalwarp" = TRUE)
• default for gdalMask has changed default for "touches". Now has equivalent for terra::mask(..., touches = TRUE), using "-wo CUTLINE_ALL_TOUCHED=TRUE"
• gdalProject now uses 2 threads, setting "-wo NUM_THREADS=2"; can be changed by user with options("reproducible.gdalwarpThreads" = X); see ?reproducibleOptions
• gdal* functions now address datatype issues
• gdal* defaults to FLT8S if datatype not passed
• makeRelative, makeAbsolute and similar have been created to ease many issues encountered in preProcess

Changes

• showSimilar (e.g., options(reproducible.showSimilar = 1)) now preferentially shows the most recent item in cache if there are several with equivalent matching.
• overhaul of messaging in Cache and prepInputs families; functions are highlighted with a different colour; indent level reflects nesting of both Cache and prepInputs, so it is easier to identify which message goes with which function call.
• preProcess is a lot faster now for large numbers of files; uses CHECKSUMS more effectively and fewer times
• retry now captures its expr so it doesn't need a quote; is like try now.
• showSimilar mechanisms now returns the most recent, if there are >1 similar that are equivalently similar
• if a user is having troubles with googledrive for e.g., large files on spotting connections, instructions for using gdown are provided
• showCache, clearCache now have extra arguments fun, cacheId, and ... now can take any arbitrary tag = value pair. The cacheId argument will be very fast if a user is not using useDBI() is FALSE.
• .wrap and .unwrap can now deal with SpatVectorCollection (a terra class that does not have a wrap/unwrap method in terra)
• ALTREP digesting when using spooky or fastdigest were not stable for integers and factors. There is now a work around in .robustDigest that stabilizes these by expanding them from their ALTREP representation first. Since they will be saved and recovered anyway, this will have little effect.
• .wrap and .unwrap are becoming more mature and can handle many more classes effectively. Methods can still be written, if needed.

Testing

• lots of testing with cacheSaveFormat = "qs", which previously was not reliable especially for environments. With all recent changes to .wrap and .unwrap, these appear stable now and should be able to be used for environments.

Bugfixes

• switchDataType can now correctly switch between gdal formats and terra
• many messaging fixes that were imprecise or missing

v2.0.12

reproducible 2.0.12

• re-submission after removal from CRAN

v2.0.9

reproducible 2.0.9

Enhancements

• new function isUpdated() to determine whether a cached object has been updated;
• makeRelative() is now exported for use downstream (e.g., SpaDES.core);
• new functions getRelative() and normPathRel() for improved symlink handling (#362);
• messaging is improved for Cache with the function named instead of just cacheId
• messaging for prepInputs: minor changes
• more edge cases for Checksums dealt with, so fewer unneeded downloads
• wrapSpatRaster (wrap for file-backed spatRaster objects) fixes for more edge cases
• postProcessTo can now use sf::gdal_utils for the case of from is a gridded object and to is a polygon vector. This appears to be between 2x and 10x faster in tests.
• postProcessTo does a pre-crop (with buffer) to make the projectTo faster. When both from and to are vector objects, this pre-crop appears to create slivers in some cases. This step is now skipped for these cases.
• Cache can now deal with unnamed functions, e.g., Cache((function(x) x)(1)). It will be refered to as "headless".
• terra would fail if internet was unavailable, even when internet is not necessary, due to needing to retrieve projection information. Many cases where this happens will now divert to use sf.
• Cache can now skip calculating objSize, which can take a non-trivial amount of time for large, complicated objects; see reproducibleOptions()

Bug fixes

• Filenames for some classes returned ""; now returns NULL so character vectors are only pointers to files
• Cache on a terra object that writes file to disk, when quick argument is specified was failing, always creating the same object; fixed with #PR368
• useDBI was incorrectly used if a user had set the option prior to package loading. Now works as expected.
• several other minor
• preProcess deals better with more cases of nested paths in archives.
• more edge cases corrected for inputPaths

v2.0.8

reproducible 2.0.8

Enhancements

• minor formatting changes

Bug fixes

• only use character strings when comparing getRVersion() <= "XXX"
• fixes for assessDataType for categorical (factor) Raster and SpatRaster

v2.0.7

reproducible 2.0.7

Enhancements

• Address change in round with R > 4.3.1; now a primitive, that does method dispatch. Failure was identified with unit tests, by Luke Tierney who was making the change in base::round.

Bug fixes

• several identified and fixed (PRs by Ceres Barros, notably, PRs #341, #342, #343). These fix missing argument in a .unwrap call, and missing check in preProcess, when targetFilePath was NULL.
• minor documentation updates

v2.0.5

reproducible 2.0.5

Enhancements

• Updates of Copy & new .wrap, .unwrap generics and methods to wrap classes that don't save well to disk as is. This uses the name similar to terra::wrap, but with slight differences internally to allow for SpatRaster objects who are file-backed and must have their files moved when they are unwrapped.
• loadFiles updated for more cases
• convert to using withr throughout testing for cleaning up
• more methods for Filename added, including for Path class
• Cache(..., useCloud = TRUE) had many cases that were not working; known cases are now working. Also, now file from file-backed cases are now placed inside the cacheOutputs folder rather than inside a separate folder (used to be "rasters")

Bugfixes

• several small for edge cases

Dependency changes

• none

v2.0.4

reproducible 2.0.4

Enhancements

• reproducible.useFuture now defaults to "multisession"
• updated tests to deal with data.table development branch (#314)
• removed all use of data.table::setattr to deal with "modified compiler constants" issue that was detected during CRAN checks at https://github.com/kalibera/cran-checks/blob/master/rcnst/results/reproducible/00check.log

Bugfixes

• preProcess failed when googledrive url filename could be found, but destinationPath was not "."
• normPath had different behaviour on *nix-alikes and Windows. Now it is the same.
• Issue #316 -- SpatRaster objects if saved to a specific, non relative (to getwd()) path would not be recovered correctly.
• Several other Issues that addressed edge cases for prepInputs and family.

Continuous integration

• Improvements with testing on GitHub Actions

v2.0.2

Known issues: https://github.com/PredictiveEcology/reproducible/issues

Version 2.0.2

Enhancements

• new optional backend for Cache via options(reproducible.useDBI = FALSE) is single data files with the same basename as the cached object, i.e., with the same cacheId in the file name. This is a replacement for RSQLite and will likely become the default in the next release. This approach makes cloud caching easier as all metadata are available in small binary files for each cached object. This is simpler, faster and creates far fewer package dependencies (now 11 recursive; before 27 recursive). If a user has DBI and RSQLite installed, then the backend will default to use these currently, i.e., the previous behaviour. The user can change the backend without loss of Cache data.
• moved raster and sp to Suggests; no more internal functions use these. User can still work with Raster and sp class objects as before.
• preProcess can now handle google docs files, if type = ... is passed.
• postProcess now uses terra and sf internally (with #253) throughout the family of postProcess functions. The previous *Input and *Output functions now redirect to the new *To* functions. These are faster, more stable, and cover vastly more cases than the previous *Inputs family. The old backends no longer work as before.
• minor functions to assist with transition from raster to terra: maxFn, minFn, rasterRead
• .dealWithClass and .dealWithClassOnRecovery are now exported generics, with several methods here, notably, list, environment, default
• other miscellaneous changes to deal with raster to terra transition (e.g. studyAreaName can deal with SpatVector)
• prepInputs now deals with archives that have sub-folder structure are now dealt with correctly in all examples and tests esp. #181.
• prepInputs can now deal with .gdb files. Though, it is limited to sf out of the box, so e.g., Raster layers inside gdb files are not supported (yet?). User can pass fun = NA to not try to load it, but at least have the .gdb file locally on disk.
• hardLinkOrCopy now uses linkOrCopy(symlink = FALSE); more cases dealt with especially nested directory structures that do not exist in the to.
• many GitHub issues closed after transition to using terra and sf.
• preProcess had multiple changes. The following now work: archives with subfolders, archives with subfolders with identical basenames (different dirnames), gdb files, other files where targetFile is a directory.
• ~40 issues were closed with current release.
• code coverage now approaching 85%
• substantial changes to preProcess for minor efficiency gains, edge cases, code cleaning
• new function CacheGeo that weaves together prepInputs and Cache to create a geo-spatial caching. See help and examples.
• maskTo now allows touches arg for terra::mask
• Spatial class is also "fixed" in fixErrorsIn
• prepInputs and preProcess now capture dlFun, so user can pass unquoted dlFun
• Copy method for SpatRaster, with and without file-backing
• Cache(..., useCloud = TRUE) reworked so appears to be more robust than previously.
• maskTo now works even if to is larger than from
• netCDF works with prepInputs; thanks to user nbsmokee with PR #300.

Dependency changes

• no spatial packages are automatically installed any more; to work with prepInputs and family, the user will have to install terra and sf at a minimum.
• terra, sf are in Suggests
• removed entirely: fasterize, fpCompare, magrittr
• moved to Suggests: raster, sp, rlang
• A normal (minimal) install of reproducible no longer installs DBI, nor does it use RSQLite. All cache repositories database files will be in binary individual files in the cacheOutputs file. If a user has DBI and a SQLite engine, then the previous behaviour will be used.

Defunct

• reproducible.useNewDigestAlgorithm is not longer an option as the old algorithms do not work reliably.

Defunct and removed

• removed assessDataTypeGDAL(), clearStubArtifacts(),
• removed non-exported digestRasterLayer2(); evalArgsOnly(); .getSourceURL(); .getTargetCRS(); .checkSums(), .groupedMessage(); .checkForAuxililaryFiles()
• option("reproducible.polygonShortcut") removed

Non exported function changes

• .basename renamed to basename2

Bugfixes

• Cache was incorrectly dealing with environment and environment-like objects. Since some objects, e.g., Spat* objects in terra, must be wrapped prior to saving, environments must be scanned for these classes of objects prior to saving. This previously only occurred for list objects;
• When working with revdep SpaDES.core, there were some cases where the Cache was failing as it could not find the module name;
• during transition from postProcess (using raster and sp) to postProcessTo, some cases are falling through the cracks; these have being addressed.

v1.2.16

Known issues: https://github.com/PredictiveEcology/reproducible/issues

Version 1.2.16

Dependency changes

• none

Enhancements

• Cache now captures the first argument passed to it without evaluating it, so Cache(rnorm(1)) now works as expected.
• As a result of previous, Cache now works with base pipe |> (with R >= 4.1).
• Due to some internal changes in the way arguments are evaluated and digested, there may be some cache entries that will be rerun. However, in simple cases of FUN passed to Cache, there should be no problems with previous cache databases being successfully recovered.
• Added more unit tests
• Reworked Cache internals so that digesting is more accurate, as the correct methods for functions are more accurately found, objects within functions are more precisely evaluated.
• Improved documentation:
  • Examples were reworked, replaced, improved;
  • All user-facing exported functions and methods now have complete documentation;
  • Added () in DESCRIPTION for functions;
  • Added \value in .Rd files for exported methods (structure, the class, the output meaning);
  • Remove commented code in examples.

Bug fixes

• postProcess now also checks resolution when assessing whether to project
• prepInputs has an internal Cache call for loading the object into memory; this was incorrectly evaluating all files if there were more than one file downloaded and extracted. This resulted in cases, e.g. shapefiles, being considered identical if they had the identical geometries, even if their data were different. This is fixed now as it uses the digest of all files extracted.

Deprecated and defunct

• remove defunct argument digestPathContent from Cache
• options("reproducible.useGDAL") is now deprecated; the package is moving towards terra.

v1.2.11

Known issues: https://github.com/PredictiveEcology/reproducible/issues

Version 1.2.11

Dependency changes

• none

Enhancements

• none

Bug fixes

• fix tests for postProcessTerra to deal with changes in GDAL/PROJ/GEOS (#253; @rsbivand)
• fixed issue with masking