Export open_mfdataset #470

meggart · 2024-11-22T14:59:11Z

This documents and export the open_mfdataset function. Also, in conjunction with the recent YAXArrayBase PR this significantly decreases times for loading stacks of NetCDF files. Opening a stack of 50 NetCDFs went from 18 -> 0.8 seconds with this new version.

github-actions · 2024-11-25T08:05:43Z

Benchmark result

Judge result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmarks:
- Target: 25 Nov 2024 - 08:05
- Baseline: 25 Nov 2024 - 08:05
Package commits:
- Target: 1d4699
- Baseline: 19db33
Julia commits:
- Target: 8f5b7c
- Baseline: 8f5b7c
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["mapslices", "small"]`	0.73 (5%) ✅	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Target

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        772 s          0 s         91 s       1780 s          0 s
       #2     0 MHz        698 s          0 s         84 s       1867 s          0 s
       #3     0 MHz        919 s          0 s        105 s       1614 s          0 s
       #4     0 MHz        809 s          0 s        112 s       1712 s          0 s
  Memory: 15.606491088867188 GB (13599.3984375 MB free)
  Uptime: 266.71 sec
  Load Avg:  2.14  1.06  0.41
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        891 s          0 s         95 s       1894 s          0 s
       #2     0 MHz        758 s          0 s         86 s       2042 s          0 s
       #3     0 MHz        946 s          0 s        108 s       1821 s          0 s
       #4     0 MHz        835 s          0 s        115 s       1920 s          0 s
  Memory: 15.606491088867188 GB (13655.87890625 MB free)
  Uptime: 290.47 sec
  Load Avg:  1.81  1.05  0.43
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmark: 25 Nov 2024 - 8:5
Package commit: 1d4699
Julia commit: 8f5b7c
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["mapslices", "small"]`	230.967 ms (5%)	148.842 ms	330.26 MiB (1%)	91275

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        772 s          0 s         91 s       1780 s          0 s
       #2     0 MHz        698 s          0 s         84 s       1867 s          0 s
       #3     0 MHz        919 s          0 s        105 s       1614 s          0 s
       #4     0 MHz        809 s          0 s        112 s       1712 s          0 s
  Memory: 15.606491088867188 GB (13599.3984375 MB free)
  Uptime: 266.71 sec
  Load Avg:  2.14  1.06  0.41
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmark: 25 Nov 2024 - 8:5
Package commit: 19db33
Julia commit: 8f5b7c
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["mapslices", "small"]`	314.731 ms (5%)	175.506 ms	331.73 MiB (1%)	121051

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        891 s          0 s         95 s       1894 s          0 s
       #2     0 MHz        758 s          0 s         86 s       2042 s          0 s
       #3     0 MHz        946 s          0 s        108 s       1821 s          0 s
       #4     0 MHz        835 s          0 s        115 s       1920 s          0 s
  Memory: 15.606491088867188 GB (13655.87890625 MB free)
  Uptime: 290.47 sec
  Load Avg:  1.81  1.05  0.43
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`lbt`
`Sys.CPU_THREADS`	4

lscpu output:

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7763 64-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4890.85
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          64 KiB (2 instances)
L1i cache:                          64 KiB (2 instances)
L2 cache:                           1 MiB (2 instances)
L3 cache:                           32 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Cpu Property	Value
Brand	AMD EPYC 7763 64-Core Processor
Vendor	:AMD
Architecture	:Unknown
Model	Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores	16 physical cores, 16 logical cores (on executing CPU)
	No Hyperthreading hardware capability detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 512, 32768) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 48 bits physical
SIMD	256 bit = 32 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

github-actions · 2024-11-25T08:16:15Z

Benchmark result

Judge result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmarks:
- Target: 25 Nov 2024 - 08:15
- Baseline: 25 Nov 2024 - 08:16
Package commits:
- Target: 0709d2
- Baseline: 19db33
Julia commits:
- Target: 8f5b7c
- Baseline: 8f5b7c
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["mapslices", "small"]`	0.74 (5%) ✅	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Target

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        699 s          0 s         84 s       1976 s          0 s
       #2     0 MHz        978 s          0 s        105 s       1661 s          0 s
       #3     0 MHz        768 s          0 s        110 s       1879 s          0 s
       #4     0 MHz        763 s          0 s         95 s       1901 s          0 s
  Memory: 15.606491088867188 GB (13619.64453125 MB free)
  Uptime: 278.31 sec
  Load Avg:  2.12  1.03  0.41
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        795 s          0 s         86 s       2118 s          0 s
       #2     0 MHz       1006 s          0 s        108 s       1870 s          0 s
       #3     0 MHz        801 s          0 s        113 s       2083 s          0 s
       #4     0 MHz        841 s          0 s        100 s       2058 s          0 s
  Memory: 15.606491088867188 GB (13605.8046875 MB free)
  Uptime: 302.34 sec
  Load Avg:  1.73  1.03  0.42
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmark: 25 Nov 2024 - 8:15
Package commit: 0709d2
Julia commit: 8f5b7c
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["mapslices", "small"]`	230.197 ms (5%)	148.520 ms	330.26 MiB (1%)	91275

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        699 s          0 s         84 s       1976 s          0 s
       #2     0 MHz        978 s          0 s        105 s       1661 s          0 s
       #3     0 MHz        768 s          0 s        110 s       1879 s          0 s
       #4     0 MHz        763 s          0 s         95 s       1901 s          0 s
  Memory: 15.606491088867188 GB (13619.64453125 MB free)
  Uptime: 278.31 sec
  Load Avg:  2.12  1.03  0.41
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmark: 25 Nov 2024 - 8:16
Package commit: 19db33
Julia commit: 8f5b7c
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["mapslices", "small"]`	310.739 ms (5%)	172.420 ms	331.73 MiB (1%)	121051

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        795 s          0 s         86 s       2118 s          0 s
       #2     0 MHz       1006 s          0 s        108 s       1870 s          0 s
       #3     0 MHz        801 s          0 s        113 s       2083 s          0 s
       #4     0 MHz        841 s          0 s        100 s       2058 s          0 s
  Memory: 15.606491088867188 GB (13605.8046875 MB free)
  Uptime: 302.34 sec
  Load Avg:  1.73  1.03  0.42
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`lbt`
`Sys.CPU_THREADS`	4

lscpu output:

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7763 64-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4890.85
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          64 KiB (2 instances)
L1i cache:                          64 KiB (2 instances)
L2 cache:                           1 MiB (2 instances)
L3 cache:                           32 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Cpu Property	Value
Brand	AMD EPYC 7763 64-Core Processor
Vendor	:AMD
Architecture	:Unknown
Model	Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores	16 physical cores, 16 logical cores (on executing CPU)
	No Hyperthreading hardware capability detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 512, 32768) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 48 bits physical
SIMD	256 bit = 32 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

src/DatasetAPI/Datasets.jl

lazarusA

just close the code-block, and CI for docs should be fine.

src/DatasetAPI/Datasets.jl

github-actions · 2024-11-25T22:43:20Z

Benchmark result

Judge result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmarks:
- Target: 25 Nov 2024 - 22:42
- Baseline: 25 Nov 2024 - 22:43
Package commits:
- Target: 112d30
- Baseline: f4253c
Julia commits:
- Target: 8f5b7c
- Baseline: 8f5b7c
Julia command flags:
- Target: None
- Baseline: None
Environment variables:
- Target: None
- Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID	time ratio	memory ratio
`["mapslices", "small"]`	0.75 (5%) ✅	1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Target

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        651 s          0 s        110 s       1835 s          0 s
       #2     0 MHz        803 s          0 s         88 s       1713 s          0 s
       #3     0 MHz        931 s          0 s        108 s       1548 s          0 s
       #4     0 MHz        820 s          0 s         88 s       1694 s          0 s
  Memory: 15.606491088867188 GB (13587.3046875 MB free)
  Uptime: 262.57 sec
  Load Avg:  2.19  1.1  0.43
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        700 s          0 s        112 s       2024 s          0 s
       #2     0 MHz        853 s          0 s         94 s       1896 s          0 s
       #3     0 MHz       1017 s          0 s        110 s       1700 s          0 s
       #4     0 MHz        870 s          0 s         91 s       1880 s          0 s
  Memory: 15.606491088867188 GB (13597.546875 MB free)
  Uptime: 286.6 sec
  Load Avg:  1.78  1.09  0.45
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmark: 25 Nov 2024 - 22:42
Package commit: 112d30
Julia commit: 8f5b7c
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["mapslices", "small"]`	230.609 ms (5%)	148.082 ms	330.26 MiB (1%)	91275

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        651 s          0 s        110 s       1835 s          0 s
       #2     0 MHz        803 s          0 s         88 s       1713 s          0 s
       #3     0 MHz        931 s          0 s        108 s       1548 s          0 s
       #4     0 MHz        820 s          0 s         88 s       1694 s          0 s
  Memory: 15.606491088867188 GB (13587.3046875 MB free)
  Uptime: 262.57 sec
  Load Avg:  2.19  1.1  0.43
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Time of benchmark: 25 Nov 2024 - 22:43
Package commit: f4253c
Julia commit: 8f5b7c
Julia command flags: None
Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID	time	GC time	memory	allocations
`["mapslices", "small"]`	305.631 ms (5%)	168.920 ms	331.73 MiB (1%)	121051

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

["mapslices"]

Julia versioninfo

Julia Version 1.11.1
Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
      Ubuntu 22.04.5 LTS
  uname: Linux 6.5.0-1025-azure #26~22.04.1-Ubuntu SMP Thu Jul 11 22:33:04 UTC 2024 x86_64 x86_64
  CPU: AMD EPYC 7763 64-Core Processor: 
              speed         user         nice          sys         idle          irq
       #1     0 MHz        700 s          0 s        112 s       2024 s          0 s
       #2     0 MHz        853 s          0 s         94 s       1896 s          0 s
       #3     0 MHz       1017 s          0 s        110 s       1700 s          0 s
       #4     0 MHz        870 s          0 s         91 s       1880 s          0 s
  Memory: 15.606491088867188 GB (13597.546875 MB free)
  Uptime: 286.6 sec
  Load Avg:  1.78  1.09  0.45
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads	2
`BLAS.vendor()`	`lbt`
`Sys.CPU_THREADS`	4

lscpu output:

Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      48 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             4
On-line CPU(s) list:                0-3
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 7763 64-Core Processor
CPU family:                         25
Model:                              1
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4890.85
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization:                     AMD-V
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          64 KiB (2 instances)
L1i cache:                          64 KiB (2 instances)
L2 cache:                           1 MiB (2 instances)
L3 cache:                           32 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Cpu Property	Value
Brand	AMD EPYC 7763 64-Core Processor
Vendor	:AMD
Architecture	:Unknown
Model	Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores	16 physical cores, 16 logical cores (on executing CPU)
	No Hyperthreading hardware capability detected
Clock Frequencies	Not supported by CPU
Data Cache	Level 1:3 : (32, 512, 32768) kbytes
	64 byte cache line size
Address Size	48 bits virtual, 48 bits physical
SIMD	256 bit = 32 byte max. SIMD vector size
Time Stamp Counter	TSC is accessible via `rdtsc`
	TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring	Performance Monitoring Counters (PMC) are not supported
Hypervisor	Yes, Microsoft

meggart added 3 commits November 22, 2024 15:45

upgrade and export open_mfdataset

e557b57

use newest YAXArrayBase

01f2370

Fix unrelated test

ce9d236

add dependabot

3354f90

lazarusA reviewed Nov 25, 2024

View reviewed changes

src/DatasetAPI/Datasets.jl Outdated Show resolved Hide resolved

lazarusA requested changes Nov 25, 2024

View reviewed changes

lazarusA reviewed Nov 25, 2024

View reviewed changes

src/DatasetAPI/Datasets.jl Outdated Show resolved Hide resolved

lazarusA added 2 commits November 25, 2024 23:29

Apply suggestions from code review

7458386

fix locally

baeb7d3

lazarusA approved these changes Nov 25, 2024

View reviewed changes

meggart merged commit d7fd921 into master Nov 26, 2024
10 checks passed

This was referenced Dec 16, 2024

slow open_dataset for large NetCDF files #460

Closed

Feature request: dataset over multiple NetCDF files. #461

Closed

Opening a set of NetCDF fails #365

Closed

KeyError: key :Ti not found #392

Closed

Export open_mfdataset #470

Export open_mfdataset #470

Conversation

meggart commented Nov 22, 2024

github-actions bot commented Nov 25, 2024

Judge result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

github-actions bot commented Nov 25, 2024

Judge result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information

lazarusA left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 25, 2024

Judge result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Target

Baseline

Target result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Baseline result

Benchmark Report for /home/runner/work/YAXArrays.jl/YAXArrays.jl

Job Properties

Results

Benchmark Group List

Julia versioninfo

Runtime information