CHANGE: switch to uv for deps/CI, pooch for data retrieval (#10)

* switch to uv and pooch to fix failing tests

* update to deps

* removing the notion of models from ilamb

* CI changes

.github/workflows/ci.yml

@@ -26,29 +26,24 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
         os: [ubuntu] #, macos, windows]
     steps:
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0 # Fetch all history for all branches and tags.
 
-      - name: Create conda environment
-        uses: mamba-org/setup-micromamba@v1
-        with:
-          cache-downloads: true
-          environment-file: ci/environment.yml
-          environment-name: ilamb3-dev
-          create-args: >-
-            python=${{ matrix.python-version }}
+      - name: Install uv environment
+        uses: astral-sh/setup-uv@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        run: uv python install ${{ matrix.python-version }}
 
-      - name: Install ilamb3
-        run: |
-          python -m pip install -e . --no-deps --force-reinstall
+      - name: Install ilamb3 and dependencies
+        run: uv sync
 
       - name: Run Tests
-        run: |
-          python -m pytest
+        run: uv run pytest
 
       - name: Upload coverage reports to Codecov
         uses: codecov/codecov-action@v4.0.1

.readthedocs.yml

@@ -1,7 +1,12 @@
 version: 2
-conda:
-  environment: ci/environment-docs.yml
+
 build:
-  os: "ubuntu-20.04"
+  os: "ubuntu-24.04"
   tools:
-    python: "mambaforge-4.10"
+    python: "3.12"
+  commands:
+    - asdf plugin add uv
+    - asdf install uv latest
+    - asdf global uv latest
+    - uv sync --group doc
+    - uv run -m sphinx -T -b html -d docs/_build/doctrees -D language=en docs $READTHEDOCS_OUTPUT/html

docs/jupyter.md

@@ -12,22 +12,20 @@ kernelspec:
 `ilamb3` has been redesigned to allow you to import our analysis functions and run them locally on your own datasets. This means that you can apply our analysis methods in your own Jupyter notebooks and python scripts. First, we import the functionality that we will need.
 
 ```{code-cell}
-import intake
 import matplotlib.pyplot as plt
-
+import xarray as xr
+import ilamb3
 from ilamb3.analysis import bias_analysis
 ```
 
 ILAMB analysis functions are available in the `ilamb3.analysis` package. You can import just this package and browse the member functions to see what is available. In this example, we will run the ILAMB bias methodology and so we import only this function. The ILAMB analysis functions have been redesigned to take as inputs two xarray datasets, a reference and a comparison. In this example, we will load two of our biomass reference data products and use the ILAMB bias methodology to compare them.
 
-ILAMB reference datasets are available through an [intake](https://github.com/intake/intake) catalog. To use it, you only need to install the `intake` package and then add the following call to `open_catalog()`. We will use the catalog to load the biomass products from [Xu & Saatchi, 2021](https://zenodo.org/records/4161694) and [ESACCI](https://climate.esa.int/en/projects/biomass/).
+ILAMB reference datasets are available through an [pooch](https://github.com/fatiando/pooch) registry available via `ilamb3.open_catalog()`. We will use the catalog to load the biomass products from [Xu & Saatchi, 2021](https://zenodo.org/records/4161694) and [ESACCI](https://climate.esa.int/en/projects/biomass/).
 
 ```{code-cell}
-cat = intake.open_catalog(
-    "https://raw.githubusercontent.com/nocollier/intake-ilamb/main/ilamb.yaml"
-)
-ds_xusaatchi = cat["biomass | XuSaatchi2021"].read()
-ds_esacci = cat["biomass | ESACCI"].read()
+cat = ilamb3.ilamb_catalog()
+ds_xusaatchi = xr.open_dataset(cat.fetch("biomass/XuSaatchi2021/XuSaatchi.nc"))
+ds_esacci = xr.open_dataset(cat.fetch("biomass/ESACCI/biomass.nc"))
 ```
 
 Now that this data is loaded into memory as xarray datasets, we can initialize an ILAMB bias analysis.

ilamb3/__init__.py

@@ -2,8 +2,10 @@
 
 # import these packages so that units via pint will be possible once anything
 # from ilamb is imported.
-import intake
+import importlib
+
 import pint_xarray  # noqa
+import pooch
 import xarray as xr
 from cf_xarray.units import units
 
@@ -15,18 +17,29 @@
 units.define("Pg = 1e15 * g")
 
 
-def ilamb_catalog() -> intake.Catalog:
+def ilamb_catalog() -> pooch.Pooch:
     """
-    Return the intake ilamb reference data catalog.
+    Return the pooch ilamb reference data catalog.
 
     Returns
     -------
-    intake.Catalog
+    pooch.Pooch
         The intake ilamb reference data catalog.
     """
-    return intake.open_catalog(
-        "https://raw.githubusercontent.com/nocollier/intake-ilamb/main/ilamb.yaml"
+
+    _ILAMB_DATA_VERSION = (
+        "0.1"  # we don't really have data versions for the collection :/
+    )
+    registry = pooch.create(
+        path=pooch.os_cache("ilamb3"),
+        base_url="https://www.ilamb.org/ILAMB-Data/DATA",
+        version=_ILAMB_DATA_VERSION,
+        env="ILAMB_ROOT",
+    )
+    registry.load_registry(
+        importlib.resources.open_binary("ilamb3.registry", "ilamb.txt")
     )
+    return registry
 
 
 __all__ = ["dataset", "compare", "analysis", "regions", "ilamb_catalog"]

ilamb3/registry/ilamb.txt

@@ -0,0 +1,121 @@
+hfss/FLUXNET2015/hfss.nc sha1:a4952cbd4735c8a13e53ee979bfe382e3db09034
+hfss/WECANN/hfss.nc sha1:66aef4700703b28a5a18e8454757e0e2dbe67a17
+hfss/FLUXCOM/sh.nc sha1:6e7a58c02b4199e857f2512092ecedce0bdd6777
+hfss/FLUXCOM/hfss.nc sha1:dfce020d31f45ccc82fcd405351558a080fff87b
+hfss/CLASS/hfss.nc sha1:0194944c11edb5adc2b745cf826c95261f536d50
+permafrost/Obu2018/Obu2018.nc sha1:fdf77f0802606b9aec3e3c9e67b8118d6bd3ae6b
+permafrost/Brown2002/Brown2002.nc sha1:dbb8c8900b9f5a2b87d36292118182e5ecfa3846
+mrro/Dai/runoff.nc sha1:08a23c5ea69fa439f296a6e6119598851453b3dd
+mrro/Dai/basins_0.5x0.5.nc sha1:172ff6f92a4036c186a6e2ecb66a678c578d0672
+mrro/LORA/LORA.nc sha1:72bb16787877591d0c54a36d74697d0d208f985a
+mrro/CLASS/mrro.nc sha1:5283142fcb2eea1e9c5dbc9d324c62496b0d6fa7
+gpp/FLUXNET2015/gpp.nc sha1:e4918f6ce9f1266e1041e83d485c9f04a0e0098d
+gpp/WECANN/gpp.nc sha1:6e864a6ae201195cdf995a3a81720188af441e13
+gpp/FLUXCOM/gpp.nc sha1:f661eb295adf9f7ad0f0531dad26c702a7988339
+reco/FLUXNET2015/reco.nc sha1:7eebb1d25b512a2d9c89e38a777b9aadbed89fa8
+reco/FLUXCOM/reco.nc sha1:8f03a11eb446d4bf1d74ae3cee849f94fb2f54f6
+rlns/FLUXNET2015/rlns.nc sha1:c2922cd6441d00e0970bdd59cf6b89e8fa3ebfbb
+rlns/CERESed4.2/rlns.nc sha1:0bd830d3622fb5c618764539859f8006b261f8ed
+rlns/GEWEX.SRB/rlns_0.5x0.5.nc sha1:bff7fd6f82283b808a815ce8c7e432bbc34f52ef
+rlns/WRMC.BSRN/rlns.nc sha1:5315c136b74bbe7d019541e15d23bd382a9fb0cb
+hfdsl/CLASS/hfdsl.nc sha1:3369114630b4a467e80880f8a503aca2bc9a2954
+rlus/FLUXNET2015/rlus.nc sha1:f116cf7a8ff2c84f365cf490a07687ba09b89b4f
+rlus/CERESed4.2/rlus.nc sha1:c8c79a78cff7d16eb55dba7b809936d42b9eb0c2
+rlus/GEWEX.SRB/rlus_0.5x0.5.nc sha1:b71bbc8269282dae3f2c546d34da95a93d3ab4dc
+rlus/WRMC.BSRN/rlus.nc sha1:9217536a13e44e0e5012dbe66d798d2df7885c41
+hfls/FLUXNET2015/hfls.nc sha1:34307d15a125c476df19eb89f79ac4fb6423432b
+hfls/WECANN/hfls.nc sha1:027f5db47d36c14b727708503e3248c53b1a7679
+hfls/FLUXCOM/le.nc sha1:d71867da50b0a74c68946deebc4bdd5cb006a32e
+hfls/FLUXCOM/hfls.nc sha1:5f289bd0f1fa0539f6df80f597ecf9d653f451d7
+hfls/CLASS/hfls.nc sha1:6ca3ffbacb83cda44507502fc969bb83fca79f99
+biomass/Thurner/biomass_0.5x0.5.nc sha1:d76bb837865b8bb4ed42ddd04a20454ed68db9bb
+biomass/XuSaatchi2021/XuSaatchi.nc sha1:244543e731dfd46f9842d6e40f4a2eeead122ee3
+biomass/USForest/biomass_0.5x0.5.nc sha1:36526e2b0411c8cabe94bc6e7a59f241c5c43edd
+biomass/ESACCI/biomass.nc sha1:2eeac984738fc17e8b6cdf0564fff192a4a08718
+biomass/Saatchi2011/biomass_0.5x0.5.nc sha1:735ef34768cadd3a47b74139ec130b4433740c03
+biomass/NBCD2000/biomass_0.5x0.5.nc sha1:7f4681448e156de8155e7708b35e4cd4dee5f738
+biomass/GEOCARBON/biomass.nc sha1:369235319774f273e376a9e7e3ffd665e5d1c83d
+evspsbl/MOD16A2/et.nc sha1:50aa114cb720cf03a124874bbba57c7b56aad99c
+evspsbl/MODIS/et_0.5x0.5.nc sha1:54b5e5a9cbc16d1d9e5330e0c5a23ee689f5fb41
+evspsbl/GLEAMv3.3a/et.nc sha1:5aaf73949af8c6b509ef16f684aa8efeccd983e2
+evspsbl/DOLCE/DOLCE.nc sha1:1eea9627efde39e31a00774b5e164be20a2f4457
+rns/FLUXNET2015/rns.nc sha1:cd08fa025133ce213b6b81c8a727180d1247c368
+rns/CERESed4.2/rns.nc sha1:0d4d38f23c98460fc17c36221e2d4ed6120865db
+rns/GEWEX.SRB/rns_0.5x0.5.nc sha1:ff990905204c1ec5ba21269597fecc96f0f20480
+rns/WRMC.BSRN/rns.nc sha1:e3e41c6595b21349c0c7637c9b9471bb6d9536ed
+rns/CLASS/rns.nc sha1:30a482549336a93c238823d801f782afd37de9ad
+tas/FLUXNET2015/tas.nc sha1:000e9b9070fc3ab830426c8922c4622cf6adb2b0
+tas/CRU4.02/tas.nc sha1:2674da18a1a93483b50b1626e7a7ab741bf53d09
+ch4/FluxnetANN/FCH4_F_ANN_monthly_wetland_tier1.nc sha1:65446c36d2c9e7dacd5d2dc770b8ed0e8ab7492d
+tasmax/CRU4.02/tasmax.nc sha1:7ad4945bdcc93c633580b52f9e0531cd4b91f827
+tasmin/CRU4.02/tasmin.nc sha1:7135fc723baa4954e1dfdc176ae17642a0ad0a2e
+fBNF/DaviesBarnard/fBNF_0.5x0.5.nc sha1:1259ac6fbd036df812a994c475d444b3f78895af
+burntArea/GFED4.1S/burntArea.nc sha1:cf9d73c6a8bfc594737c9ba6ca4df613df4a28ab
+mrsos/WangMao/mrsos_olc.nc sha1:23082e5776ffd3a7d6d00e160dc9d91d06321c13
+nbp/GCP/nbp_1959-2016.nc sha1:1f0ba5517e67c3820ac216893ea22b152fba9636
+nbp/HOFFMAN/nbp_1850-2010.nc sha1:8350af00614d6afc6b70ad314aa499a9ece80ec2
+cSoil/Wang2024/wang2024_cSoil.nc sha1:aa0121a465cb631ca15b47a39f5fc581522e708c
+cSoil/HWSD/soilc_0.5x0.5.nc sha1:5dd084a66a7e0d0b4ca6125db47af0eb0c119377
+cSoil/HWSD2/hwsd2_cSoil.nc sha1:9a6377e4c5ff457c08c194d2c376c46e003a4f84
+cSoil/NCSCDV22/soilc_0.5x0.5.nc sha1:605efa69f547277309eca87cb581374dd3a76000
+cSoil/Koven/fracpeat_0.5x0.5.nc sha1:fa7cab1379c66baf7c1977d3c4c676ee9678de49
+cSoil/Koven/pet_0.5x0.5.nc sha1:634353ebb15811db69aeeb752aaa348eea54a4bd
+cSoil/Koven/npp_0.5x0.5.nc sha1:9df054f37fb451214ce4aa158d34c4cbdda0a264
+rsns/FLUXNET2015/rsns.nc sha1:3a717d1b9177d03619cad4b0c3560099c9cfe99a
+rsns/CERESed4.2/rsns.nc sha1:0244b366401e7bd74d965ea6fe9c5cbef5264813
+rsns/GEWEX.SRB/rsns_0.5x0.5.nc sha1:682765021db0ae5760c21e4b6af8372eafb69b54
+rsns/WRMC.BSRN/rsns.nc sha1:9d3144848c19db632ede0cbb0639354dd4373037
+pr/CMAPv1904/pr.nc sha1:4b6479f267f1ac162c0f9f1407c08059fe060249
+pr/FLUXNET2015/pr.nc sha1:5bf04ef54f4fdc189869056690451998348c3a06
+pr/GPCPv2.3/pr.nc sha1:e1b942863ec76a75aa972b6d75e2e08646741259
+pr/GPCCv2018/pr.nc sha1:c1db21fc07bcab7470676aa34954ce1fbac469f7
+pr/CLASS/pr.nc sha1:7fc3b1146684e63c1b82b0033c32b9b8ead9646e
+albedo/CERESed4.2/albedo.nc sha1:1b8e48a24564fe7b7a20ded384fbdfa4b245eea2
+albedo/GEWEX.SRB/albedo_0.5x0.5.nc sha1:7e5ff41328e788fbc4095a94b4458e22e3da4b2b
+rlds/FLUXNET2015/rlds.nc sha1:f41721d967368aec45be94d6ab7069abdd1516d0
+rlds/CERESed4.2/rlds.nc sha1:f1d860802de398e17153e24f302d670545b0374e
+rlds/GEWEX.SRB/rlds_0.5x0.5.nc sha1:c7f627159dc73f06f5a4af097d4a84bed834cf2e
+rlds/WRMC.BSRN/rlds.nc sha1:98ce3aa113ceeee782f9bbc1d0348bc861dd1c30
+nee/FLUXNET2015/nee.nc sha1:f006f83d40418f6bf83f9e1aa94fe3f20090f537
+nee/FLUXCOM/nee.nc sha1:ac74fbb4e9ee4aaceb982ab93118c6f4eccdf93d
+twsa/GRACE/twsa_0.5x0.5.nc sha1:d87f961ca4f2bf275a68ac480e312f2740178691
+regions/Whittaker.nc sha1:46364050ab75f564cb39140c1100de70248d99ec
+regions/LandRegions.nc sha1:f29f9b312d077f2edfaee063fad3cbeaec4b379c
+regions/ABoVE.nc sha1:e37b628657c879d19b84759e78196c187e5fb796
+regions/Koppen.nc sha1:78df0cc91e143dc903c6c6c0f1901f982fb42857
+regions/IPCCRegions.nc sha1:0213a8606336bd9964959a2462f250a934200608
+regions/GlobalLand.nc sha1:2f987d44fdba6ad0e72d14d6a2fecb7e8df2a9c5
+regions/GlobalLandNoAnt.nc sha1:71668afcb0017fbe31c62000cac9bd8202426ce9
+co2/NOAA.GMD/co2.nc sha1:e1fd3d93571fd3b865a984f46ca6411106b4977b
+co2/PulseEmulation/Pulse05.nc sha1:55e752b54dc446d9a315f8c5c19fd62d213133ec
+co2/PulseEmulation/Pulse11.nc sha1:b661f8fdcfd1489ba3fe693bedac46ed99322044
+co2/PulseEmulation/Pulse03.nc sha1:1d0c2303875855e9ecf715ee8ee2c6631acccf8e
+co2/PulseEmulation/Pulse08.nc sha1:65c71dafe6e1abd3209d90d1611284ce4fb3d39d
+co2/PulseEmulation/Pulse10.nc sha1:72854ac1834577a94deccc37b580b32030ce3de3
+co2/PulseEmulation/Pulse04.nc sha1:edd28b2a66765f96f1070a7cf07bb12039b5218e
+co2/PulseEmulation/Pulse06.nc sha1:e4aece52fc70f8b3d65a5506fa940d336307c80c
+co2/PulseEmulation/Pulse01.nc sha1:d6c1712c3bb0771860f9a1d3896d7f48a7d32b5b
+co2/PulseEmulation/Pulse02.nc sha1:92ce57716665e306f6d3b02056f8824dde6db44a
+co2/PulseEmulation/AtmosphericPulseRegions.nc sha1:06fd8284b752b782d70bd0c7c46c534a86b1dd18
+co2/PulseEmulation/Pulse12.nc sha1:7c327541c32ed008a39cb46111083bdaa8474f72
+co2/PulseEmulation/Pulse09.nc sha1:7a6d3108ac76db42655dfe1e99da7ef82e5b2fec
+co2/PulseEmulation/Pulse07.nc sha1:304c17c7c62171361e16a956a683eabb0e66bf72
+co2/PulseEmulation/GEOSChemOcnFfCo2_32yr_360daytime.nc sha1:86ff8718aee41fd1251d731199524609721bac95
+co2/HIPPOATom/HIPPO_ATom.nc sha1:c841ad3f84c118499067fb9c5f934620b495f05e
+active_layer_thickness/CALM/CALM.nc sha1:fe084ab58a81911e3739c8bb3fad210b3863af27
+rsds/FLUXNET2015/rsds.nc sha1:812e5c2893eae629d2a440fa2b673296ad9a48ab
+rsds/CERESed4.2/rsds.nc sha1:66c328847234e7d1586eb810f2a184f7c31cf5d3
+rsds/GEWEX.SRB/rsds_0.5x0.5.nc sha1:fcde36ae3371aa784bd0e587be320524d19bd701
+rsds/WRMC.BSRN/rsds.nc sha1:8355c203eabefa248a6edb9bcf2562e32d60aa77
+lai/GIMMS_LAI4g/cao2023_lai.nc sha1:63f54dde4158afb176175eaf054923bdbfb6240b
+lai/AVH15C1/lai.nc sha1:ccace4f84912d63acbb9ee09ee7b743412207a0d
+lai/MODIS/lai_0.5x0.5.nc sha1:77c198e1b6fc768deb374a90611726ff87f398b1
+lai/AVHRR/lai_0.5x0.5.nc sha1:8b2bdece87b97c3cef8533bbcf34d9d1d7a28fbd
+dtr/CRU4.02/dtr.nc sha1:6df77d8b941742df4c0f24c3c226ac3eebe01b09
+rsus/FLUXNET2015/rsus.nc sha1:0e8cfc972e9d6b8da3340e9e8bae73ba58119f17
+rsus/CERESed4.2/rsus.nc sha1:83a23d700148d232ce018c2eedc1149d176f5262
+rsus/GEWEX.SRB/rsus_0.5x0.5.nc sha1:f7de112d610762117b787f33da74ae7bdb8e3ac0
+rsus/WRMC.BSRN/rsus.nc sha1:d33e72f493703a0f05d7e6016b9e554d6c40362b
+swe/CanSISE/swe.nc sha1:af973dff1213aa289290ea741d8d5137dc8cee2f
+rhums/ERA5/rhums.nc sha1:2e22d185031de30f1bc073000d1e842ebc04c90d
+rhums/CRU4.02/rhums.nc sha1:8267af22eeeaafb956b95b553397e285295ef79e