Export: Add Zarr local/s3 support, Remove NetCDF s3 support (#509)

* Remove h5netcdf engine and fillvalue encoding * try setting format to NETCDF4 * typo * Use scipy engine for S3 export * Add Zarr export option * rechunk data before exporting Zarr * Fix proper data refs * Removed netcdf to S3, add Zarr locally * Try removing metadata update * Add remove_zarr helper * Add Zarr info to readthedocs * improve remove_zarr and add docstring * Add s3 option for Zarr export to readthedocs * Control mode option in export * make _convert_da_to_ds func to consolidate code * Bail on existing S3 path for Zarr export * Fix xarray Dataset type call
cal-adapt · Feb 26, 2025 · 157809b · 157809b
1 parent 5622939
commit 157809b
Show file tree

Hide file tree

Showing 3 changed files with 270 additions and 235 deletions.
diff --git a/climakitae/__init__.py b/climakitae/__init__.py
@@ -1,5 +1,5 @@
 from climakitae.core.data_load import load
-from climakitae.core.data_export import export
+from climakitae.core.data_export import export, remove_zarr
 
 try:
     from importlib.metadata import version as _version
@@ -18,6 +18,7 @@
     # Methods
     "load",
     "export",
+    "remove_zarr",
     # Constants
     "__version__",
 )