Add CachingStore

[ion-elgreco]

Was talking with @danielgafni on this. Would be nice if we can have a CachingStore, would work similarly to LimitStore, where we can wrap the inner store.

The CachingStore should be configurable with cache_size, tmp location and so forth.

I know there are a couple example implementations on this:

• Caching layer for immutable commit files https://github.com/delta-io/delta-rs/blob/5d2cf488ff7f69393b1e3c73bf72f1dcdd832768/crates/core/src/kernel/snapshot_next/cache.rs
• Bytes aware caching layer (focused on parquet reading): https://github.com/splitgraph/seafowl/blob/main/src/object_store/cache.rs

So perhaps we can add a general-purpose caching layer for immutable objects, and a version for potential mutable objects and a version for parquet specifically.

[kylebarron]

I agree the caching would be nice, and would be a logical extension of the existing functionality.

But caching gets really complex really quickly, and I don't want the full responsibility in this repo of managing that caching. Ideally I'd prefer that to be shipped as a standalone Python library, but given that object_store doesn't have stable FFI, I don't think that's really possible (at full performance).

So my suggestion would be to create a third party Rust repo (object-store-cache?), which means that that caching middleware could be used by obstore but also by any other Rust library, and then once that repo is stable, we can expose a wrapper through obstore.

[danielgafni]

I think your proposal makes sense.
I will think if I can manage this alone (my experience with Rust is pretty limited, mostly as a reader) - creating a standalone project is more complicated than contributing to an existing one.
Do any of you guys have any interest in working on this together?

[kylebarron]

I don't have near-term availability to work on caching, but I could maybe review some PRs. @ion-elgreco would you be interested in setting up the repo and doing an initial implementation, borrowing for other existing implementations?

[ion-elgreco]

I am a bit swamped in my weekends for the next 1-2 months. After that I should be able to support in dev time. In the mean time I can help with reviews though

[danielgafni]

Alright. I started working on something and will ask you guys to take a look soon :)