Skip to content

Commit

Permalink
add some docs
Browse files Browse the repository at this point in the history
  • Loading branch information
meggart committed Jul 25, 2024
1 parent 621ecc6 commit baba914
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 1 deletion.
18 changes: 18 additions & 0 deletions docs/src/UserGuide/cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Caching YAXArrays

For some applications like interactive plotting of large datasets it can not be avoided that the same data must be accessed several times. In these cases it can be useful to store recently accessed data in a cache. In YAXArrays this can be easily achieved using the `cache` function. For example, if we open a large dataset from a remote source and want to keep data in a cache of size 500MB one can use:

````julia
using YAXArrays, Zarr
ds = open_dataset("path/to/source")
cachesize = 500 #MB
cache(ds,maxsize = cachesize)
````

The above will wrap every array in the dataset into its own cache, where the 500MB are distributed equally across datasets.
Alternatively individual caches can be applied to single `YAXArray`s

````julia
yax = ds.avariable
cache(yax,maxsize = 1000)
````
2 changes: 1 addition & 1 deletion src/Cubes/Cubes.jl
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ using Tables: istable, schema, columns
using DimensionalData: DimensionalData as DD, AbstractDimArray, NoName
import DimensionalData: name

export concatenatecubes, caxes, subsetcube, readcubedata, renameaxis!, YAXArray, setchunks
export concatenatecubes, caxes, subsetcube, readcubedata, renameaxis!, YAXArray, setchunks, cache

"""
This function calculates a subset of a cube's data
Expand Down
4 changes: 4 additions & 0 deletions test/Datasets/datasets.jl
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,10 @@ end
end

@testset "Caching" begin
using YAXArrays.Cubes.DiskArrays.TestTypes
using YAXArrays.Cubes: DiskArrays
using Test
import DimensionalData as DD
a = AccessCountDiskArray(reshape(1:100, 5, 20), chunksize=(2, 10))
ds = Dataset(; ar=YAXArray((DD.X(1:5), DD.Y(1:20)), a))
dscached = DiskArrays.cache(ds)
Expand Down

0 comments on commit baba914

Please sign in to comment.