Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no more Ti #479

Merged
merged 14 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 19 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,18 +107,20 @@ using YAXArrays
Let's assemble a `YAXArray` with 4 dimensions i.e. time, x,y and a variable dimension with two variables.

```julia
using YAXArrays, DimensionalData
using YAXArrays
using YAXArrays: YAXArrays as YAX

axlist = (
Dim{:time}(range(1, 20, length=20)),
X(range(1, 10, length=10)),
Y(range(1, 5, length=15)),
Dim{:Variable}(["var1", "var2"]))
YAX.time(range(1, 20, length=20)),
lon(range(1, 10, length=10)),
lat(range(1, 5, length=15)),
Variables(["var1", "var2"]))
```
```
↓ time 1.0:1.0:20.0,
X 1.0:1.0:10.0,
Y 1.0:0.2857142857142857:5.0,
Variable ["var1", "var2"]
lon 1.0:1.0:10.0,
lat 1.0:0.2857142857142857:5.0,
Variables ["var1", "var2"]
```

and the corresponding data.
Expand Down Expand Up @@ -148,8 +150,8 @@ ds = YAXArray(axlist, data, props)
│ 20×10×15×2 YAXArray{Float64,4} │
├────────────────────────────────┴─────────────────────────────────────────────── dims ┐
↓ time Sampled{Float64} 1.0:1.0:20.0 ForwardOrdered Regular Points,
X Sampled{Float64} 1.0:1.0:10.0 ForwardOrdered Regular Points,
Y Sampled{Float64} 1.0:0.2857142857142857:5.0 ForwardOrdered Regular Points,
lon Sampled{Float64} 1.0:1.0:10.0 ForwardOrdered Regular Points,
lat Sampled{Float64} 1.0:0.2857142857142857:5.0 ForwardOrdered Regular Points,
⬔ Variable Categorical{String} ["var1", "var2"] ForwardOrdered
├──────────────────────────────────────────────────────────────────────────── metadata ┤
Dict{String, String} with 5 entries:
Expand All @@ -168,17 +170,17 @@ ds = YAXArray(axlist, data, props)
For axis can be via `.`

```julia
ds.X
ds.lon
```
```
X Sampled{Float64} ForwardOrdered Regular Points
lon Sampled{Float64} ForwardOrdered Regular Points
wrapping: 1.0:1.0:10.0
```

or better yet via `lookup`

```julia
lookup(ds, :X)
lookup(ds, :lon)
```
```
Sampled{Float64} ForwardOrdered Regular Points
Expand All @@ -187,7 +189,7 @@ wrapping: 1.0:1.0:10.0

note that also the `.data` field can be use
```julia
lookup(ds, :X).data
lookup(ds, :lon).data
```
```
1.0:1.0:10.0
Expand All @@ -196,15 +198,15 @@ lookup(ds, :X).data
The data for one variables, i.e. `var1` can be accessed via:

```julia
ds[Variable=At("var1")]
ds[Variables=At("var1")]
```
```
╭──────────────────────────────╮
│ 20×10×15 YAXArray{Float64,3} │
├──────────────────────────────┴────────────────────────────────────────────── dims ┐
↓ time Sampled{Float64} 1.0:1.0:20.0 ForwardOrdered Regular Points,
X Sampled{Float64} 1.0:1.0:10.0 ForwardOrdered Regular Points,
Y Sampled{Float64} 1.0:0.2857142857142857:5.0 ForwardOrdered Regular Points
lon Sampled{Float64} 1.0:1.0:10.0 ForwardOrdered Regular Points,
lat Sampled{Float64} 1.0:0.2857142857142857:5.0 ForwardOrdered Regular Points
├───────────────────────────────────────────────────────────────────────── metadata ┤
Dict{String, String} with 5 entries:
"var1" => "one of your variables"
Expand Down
1 change: 0 additions & 1 deletion docs/Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ MultivariateStats = "6f286f6a-111f-5878-ab1e-185364afe411"
NetCDF = "30363a11-5582-574a-97bb-aa9a979735b9"
OnlineStats = "a15396b6-48d5-5d58-9928-6d29437db91e"
PlotUtils = "995b91a9-d308-5afd-9ec6-746e21dbc043"
Rasters = "a3a2b9e3-a471-40c9-b274-f788e487c689"
SkipNan = "aed68c70-c8b0-4309-8cd1-d392a74f991a"
Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
TimeSeries = "9e3dc215-6440-5c97-bce1-76c03772f85e"
Expand Down
12 changes: 7 additions & 5 deletions docs/src/UserGuide/combine.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ The resulting array `whole_year` still has one dimension, i.e. time, but with 12

````@example cat
using YAXArrays
using YAXArrays: YAXArrays as YAX

first_half = YAXArray((Dim{:time}(1:6),), rand(6))
second_half = YAXArray((Dim{:time}(7:12),), rand(6))
first_half = YAXArray((YAX.time(1:6),), rand(6))
second_half = YAXArray((YAX.time(7:12),), rand(6))
whole_year = cat(first_half, second_half, dims = 1)
````

Expand All @@ -25,10 +26,11 @@ Note that using a `Dataset` instead is a more flexible approach in handling diff

````@example concatenatecubes
using YAXArrays
using YAXArrays: YAXArrays as YAX

temperature = YAXArray((Dim{:time}(1:6),), rand(6))
precipitation = YAXArray((Dim{:time}(1:6),), rand(6))
temperature = YAXArray((YAX.time(1:6),), rand(6))
precipitation = YAXArray((YAX.time(1:6),), rand(6))
cubes = [temperature,precipitation]
var_axis = Dim{:variable}(["temp", "prep"])
var_axis = Variables(["temp", "prep"])
combined = concatenatecubes(cubes, var_axis)
````
50 changes: 28 additions & 22 deletions docs/src/UserGuide/compute.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,13 @@ Let's start by creating an example dataset:

````@example compute
using YAXArrays
using YAXArrays: YAXArrays as YAX
using Dates

axlist = (
Dim{:time}(Date("2022-01-01"):Day(1):Date("2022-01-30")),
Dim{:lon}(range(1, 10, length=10)),
Dim{:lat}(range(1, 5, length=15)),
YAX.time(Date("2022-01-01"):Day(1):Date("2022-01-30")),
lon(range(1, 10, length=10)),
lat(range(1, 5, length=15)),
)
data = rand(30, 10, 15)
properties = Dict(:origin => "user guide")
Expand Down Expand Up @@ -103,6 +104,7 @@ Here, we will define a simple function, that will take as input several `YAXArra

````@example mapCube
using YAXArrays, Zarr
using YAXArrays: YAXArrays as YAX
using Dates
````

Expand Down Expand Up @@ -130,23 +132,23 @@ Note the `.` after `f`, this is because we will slice across time, namely, the f
Here, we do create `YAXArrays` only with the desired dimensions as

````@ansi mapCube
lon = YAXArray(Dim{:lon}(range(1, 15)))
lat = YAXArray(Dim{:lat}(range(1, 10)))
lon_yax = YAXArray(lon(range(1, 15)))
lat_yax = YAXArray(lat(range(1, 10)))
````

And a time Cube's Axis

````@example mapCube
tspan = Date("2022-01-01"):Day(1):Date("2022-01-30")
time = YAXArray(Dim{:time}(tspan))
time_yax = YAXArray(YAX.time(tspan))
````

note that the following can be extended to arbitrary `YAXArrays` with additional data and dimensions.

Let's generate a new `cube` using `mapCube` and saving the output directly into disk.

````@ansi mapCube
gen_cube = mapCube(g, (lon, lat, time);
gen_cube = mapCube(g, (lon_yax, lat_yax, time_yax);
indims = (InDims(), InDims(), InDims("time")),
outdims = OutDims("time", overwrite=true, path="my_gen_cube.zarr", backend=:zarr,
outtype = Float32)
Expand All @@ -170,7 +172,7 @@ gen_cube.data[1, :, :]
but, we can generate a another cube with a different `output order` as follows

````@ansi mapCube
gen_cube = mapCube(g, (lon, lat, time);
gen_cube = mapCube(g, (lon_yax, lat_yax, time_yax);
indims = (InDims("lon"), InDims(), InDims()),
outdims = OutDims("lon", overwrite=true, path="my_gen_cube.zarr", backend=:zarr,
outtype = Float32)
Expand All @@ -197,15 +199,17 @@ which outputs the same as the `gen_cube.data[1, :, :]` called above.
Here, we will consider different scenarios, namely how we deal with different input cubes and how to specify the output ones. We will illustrate this with the following test example and the subsequent function definitions.

````@example outdims
using YAXArrays, Dates
using YAXArrays
using YAXArrays: YAXArrays as YAX
using Dates
using Zarr
using Random

axlist = (
Dim{:time}(Date("2022-01-01"):Day(1):Date("2022-01-05")),
Dim{:lon}(range(1, 4, length=4)),
Dim{:lat}(range(1, 3, length=3)),
Dim{:variables}(["a", "b"])
YAX.time(Date("2022-01-01"):Day(1):Date("2022-01-05")),
lon(range(1, 4, length=4)),
lat(range(1, 3, length=3)),
Variables(["a", "b"])
)

Random.seed!(123)
Expand Down Expand Up @@ -308,7 +312,7 @@ Here, the goal is to operate at the pixel level (longitude, latitude), and then
Random.seed!(123)
data = rand(3.0:5.0, 5, 4, 3)

axlist = (Dim{:lon}(1:4), Dim{:lat}(1:3), Dim{:depth}(1:7),)
axlist = (lon(1:4), lat(1:3), Dim{:depth}(1:7),)
yax_2d = YAXArray(axlist, rand(-3.0:0.0, 4, 3, 7))
````

Expand All @@ -318,8 +322,8 @@ and
Random.seed!(123)
data = rand(3.0:5.0, 5, 4, 3)

axlist = (Dim{:time}(Date("2022-01-01"):Day(1):Date("2022-01-05")),
Dim{:lon}(1:4), Dim{:lat}(1:3),)
axlist = (YAX.time(Date("2022-01-01"):Day(1):Date("2022-01-05")),
lon(1:4), lat(1:3),)

properties = Dict("description" => "multi dimensional test cube")
yax_test = YAXArray(axlist, data, properties)
Expand Down Expand Up @@ -359,13 +363,14 @@ First, create the raster array:

````@example compute_mapcube
using YAXArrays
using YAXArrays: YAXArrays as YAX
using DimensionalData
using Dates

axlist = (
Dim{:time}(Date("2022-01-01"):Day(1):Date("2022-01-30")),
Dim{:lon}(range(1, 10, length=10)),
Dim{:lat}(range(1, 5, length=15)),
YAX.time(Date("2022-01-01"):Day(1):Date("2022-01-30")),
lon(range(1, 10, length=10)),
lat(range(1, 5, length=15)),
)
data = rand(30, 10, 15)
raster_arr = YAXArray(axlist, data)
Expand Down Expand Up @@ -432,13 +437,14 @@ The following code does a time mean over all grid points using multiple CPUs of

````julia
using YAXArrays
using YAXArrays: YAXArrays as YAX
using Dates
using Distributed

axlist = (
Dim{:time}(Date("2022-01-01"):Day(1):Date("2022-01-30")),
Dim{:lon}(range(1, 10, length=10)),
Dim{:lat}(range(1, 5, length=15)),
YAX.time(Date("2022-01-01"):Day(1):Date("2022-01-30")),
lon(range(1, 10, length=10)),
lat(range(1, 5, length=15)),
)
data = rand(30, 10, 15)
properties = Dict(:origin => "user guide")
Expand Down
4 changes: 2 additions & 2 deletions docs/src/UserGuide/convert.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ m2 = collect(a.data)

A `Raster` as defined in [Rasters.jl](https://rafaqz.github.io/Rasters.jl/stable/) has a same supertype of a `YAXArray`, i.e. `AbstractDimArray`, allowing easy conversion between those types:

````@example convert
````julia
using Rasters

lon, lat = X(25:1:30), Y(25:1:30)
Expand All @@ -43,7 +43,7 @@ ras = Raster(rand(lon, lat, time))
a = YAXArray(dims(ras), ras.data)
````

````@example convert
````julia
ras2 = Raster(a)
````

Expand Down
8 changes: 5 additions & 3 deletions docs/src/UserGuide/create.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ We can create a new YAXArray by filling the values directly:

````@example create
using YAXArrays
using YAXArrays: YAXArrays as YAX

a1 = YAXArray(rand(10, 20, 5))
````

Expand All @@ -18,9 +20,9 @@ We can also specify the dimensions with custom names enabling easier access:
using Dates

axlist = (
Dim{:time}(Date("2022-01-01"):Day(1):Date("2022-01-30")),
Dim{:lon}(range(1, 10, length=10)),
Dim{:lat}(range(1, 5, length=15)),
YAX.time(Date("2022-01-01"):Day(1):Date("2022-01-30")),
lon(range(1, 10, length=10)),
lat(range(1, 5, length=15)),
)
data2 = rand(30, 10, 15)
properties = Dict(:origin => "user guide")
Expand Down
20 changes: 12 additions & 8 deletions docs/src/UserGuide/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,12 @@ It is possible to concatenate several cubes that shared the same dimensions usin
Let's create two dummy cubes
````@example howdoi
using YAXArrays
using YAXArrays: YAXArrays as YAX

axlist = (
Dim{:time}(range(1, 20, length=20)),
Dim{:lon}(range(1, 10, length=10)),
Dim{:lat}(range(1, 5, length=15))
YAX.time(range(1, 20, length=20)),
lon(range(1, 10, length=10)),
lat(range(1, 5, length=15))
)

data1 = rand(20, 10, 15)
Expand Down Expand Up @@ -125,7 +127,7 @@ t = Date("2020-01-01"):Month(1):Date("2022-12-31")
create YAXArray axes

```@example howdoi
axes = (Dim{:Lon}(1:10), Dim{:Lat}(1:10), Dim{:Time}(t))
axes = (Lon(1:10), Lat(1:10), YAX.Time(t))
```

create the YAXArray
Expand Down Expand Up @@ -169,7 +171,7 @@ using Dates # To generate the dates of the time axis
using DimensionalData # To use the "Between" option for selecting data

t = Date("2020-01-01"):Month(1):Date("2022-12-31")
axes = (Dim{:Lon}(1:10), Dim{:Lat}(1:10), Dim{:Time}(t))
axes = (Lon(1:10), Lat(1:10), YAX.Time(t))

var1 = YAXArray(axes, reshape(1:3600, (10, 10, 36)))
var2 = YAXArray(axes, reshape((1:3600)*5, (10, 10, 36)))
Expand All @@ -195,7 +197,7 @@ using DimensionalData # To use the "Between" selector for selecting data

t = Date("2020-01-01"):Month(1):Date("2022-12-31")
common_axis = Dim{:points}(1:100)
time_axis = Dim{:Time}(t)
time_axis = YAX.Time(t)

# Note that longitudes and latitudes are not dimensions, but YAXArrays
longitudes = YAXArray((common_axis,), rand(1:369, 100)) # 100 random values taken from 1 to 359
Expand Down Expand Up @@ -324,7 +326,9 @@ You will not be able to save this dataset, first you will need to rename those `
In this section we will use `MarketData.jl` and `TimeSeries.jl` to simulate some stocks.

````@example howdoi
using YAXArrays, DimensionalData
using YAXArrays
using YAXArrays: YAXArrays as YAX
using DimensionalData
using MarketData, TimeSeries

stocks = Dict(:Stock1 => random_ohlcv(), :Stock2 => random_ohlcv(), :Stock3 => random_ohlcv())
Expand All @@ -334,7 +338,7 @@ d_keys = keys(stocks)
currently there is not direct support to obtain `dims` from a `TimeArray`, but we can code a function for it

````@example howdoi
getTArrayAxes(ta::TimeArray) = (Dim{:time}(timestamp(ta)), Dim{:variable}(colnames(ta)), );
getTArrayAxes(ta::TimeArray) = (YAX.time(timestamp(ta)), Variables(colnames(ta)), );
nothing # hide
````
then, we create the `YAXArrays` as
Expand Down
Loading
Loading