Relation to stac-utils/stac-geoparquet

[scottyhq]

Thanks for the recent presentation on the Pangeo showcase @gadomski ! Really appreciate your work and outreach in this space. It's great to understand how various STAC tools are coming together & stacrs seems like a fantastic addition.

One piece I'm not clear on is this package's connection to https://github.com/stac-utils/stac-geoparquet (I expected it to be a dependency for parsing)? Comparing the two some differences that are immediately apparent to me are 1. stacrs uses async functions, 2. has search capabilities, 3. no dependencies (this seems magic... stac-geoparquet has many including a big one in pyarrow)!... I'm sure there are others.

I'm currently using stac-geoparquet for going from stac-client results -> geodataframe (filtering & modifying things via geopandas) -> optionally save parquet. I don't think this is possible with stacrs alone currently because of losing columns with nested dictionaries like assets and links?

Example abbreviated workflow (could stacrs replace stac_geoparquet here)?:

# stac-geoparquet  items -> geodataframe
record_batch_reader = stac_geoparquet.arrow.parse_stac_items_to_arrow(items)
gf = gpd.GeoDataFrame.from_arrow(record_batch_reader)

# do things
subset = gf.apply(custom_filters)

# stac-geoparquet geodataframe -> disk
subset.to_parquet('items.parquet')

[gadomski]

Great questions @scottyhq! I appreciate the feedback — I've under-built the documentation and context for this repo, and now seems like a good time to spend some time there. For some more context, here's the slides from my FOSS4G-NA talk last year on this ecosystem: https://www.gadom.ski/2024-09-FOSS4G-NA-stac-rs/

3. no dependencies (this seems magic... stac-geoparquet has many including a big one in pyarrow)!...

stacrs is built on https://github.com/stac-utils/stac-rs, so no Python dependencies, just Rust ones. The wheels are pre-built for most platforms, so it should be mostly painless to install from PyPI. If there's not wheels for your platform Python will try to build them locally, which will require Rust on your system.

I don't think this is possible with stacrs alone currently because of losing columns with nested dictionaries like assets and links?

This feels like a bug, if true — I do my best to support direct roundtrips from stac-geoparquet and stac-rs (see this script for the check: https://github.com/stac-utils/stac-rs/blob/main/scripts/validate-stac-geoparquet).

[scottyhq]

Ah fantastic thanks for the additional context.

support direct roundtrips from stac-geoparquet and stac-rs

Yes, I guess what I'm after is an "indirect" roundtrip, where I'm going to geopandas in between :) See this issue for more context stac-utils/stac-geoparquet#76

[gadomski]

@scottyhq just released https://pypi.org/project/stacrs/0.5.6b1/, which should let you do this:

items = ...  # Either a list or a feature collection
data_frame = GeoDataFrame.from_arrow(stacrs.to_arrow(items))
# Magic geopandas stuff
item_collection: dict[str, Any] = stacrs.from_arrow(data_frame.to_arrow())

If it works for you, I'll release it. To get the pre-release: python -m pip install --pre --upgrade stacrs.

[scottyhq]

Amazing! Thanks for working on this and adding to the example notebook. I did a quick test using an example from my last PR over in stac-geoparquet, and its working like a charm:

# Install pre-release w/ pixi:
# [tool.pixi.pypi-dependencies]
# stacrs = { version = "==0.5.6b1", extras=["arrow"]}

# Test handling messy STAC https://github.com/stac-utils/stac-geoparquet/pull/80

import urllib.request, json
import stacrs
import geopandas as gpd
import pystac

href = 'https://raw.githubusercontent.com/stac-utils/stac-geoparquet/refs/heads/main/tests/data/umbra-sar.json'
with urllib.request.urlopen(href) as url:
    items = json.load(url)

gf = gpd.GeoDataFrame.from_arrow(stacrs.to_arrow(items))
feature_collection = stacrs.from_arrow(gf.to_arrow())
# ic = pystac.ItemCollection.from_dict(feature_collection) # can also use pystac
table = stacrs.to_arrow(feature_collection) # arro3.core.Table
gf1 = gpd.GeoDataFrame.from_arrow(table)
gpd.pd.testing.assert_frame_equal(gf, gf1, check_like=True) # column order changes, otherwise identical

[gadomski]

I've added a section to the README to talk about the relationship between this repo and stac-utils/stac-geoparquet.