issue_comments: 1362562275

Just tested with to_zarr and it goes through:

State: COMPLETED (exit code 0) Nodes: 1 Cores per node: 2 CPU Utilized: 00:07:55 CPU Efficiency: 63.00% of 00:12:34 core-walltime Job Wall-clock time: 00:06:17 Memory Utilized: 164.89 GB Memory Efficiency: 44.56% of 370.00 GB

I did an extra run using a memory profiler as such:

``` import xarray as xr import zarr from memory_profiler import profile

@profile def main(): path = './data/data_*.nc' # files are: data_1.nc data_2.nc data_3.nc data_4.nc data_5.nc data = xr.open_mfdataset(path)

data = data.load()
data = data.compute()

data.to_zarr()

if name=='main': main() The profiled code was also completed with great success: State: COMPLETED (exit code 0) Nodes: 1 Cores per node: 2 CPU Utilized: 00:07:52 CPU Efficiency: 63.61% of 00:12:22 core-walltime Job Wall-clock time: 00:06:11 Memory Utilized: 165.53 GB Memory Efficiency: 44.74% of 370.00 GB ```

Here is the outcome for the memory profiling: ``` Line # Mem usage Increment Occurrences Line Contents ============================================================= 5 156.9 MiB 156.9 MiB 1 @profile 6 def main(): 7 156.9 MiB 0.0 MiB 1 path = './data/data_*.nc' # files are: data_1.nc data_2.nc data_3.nc data_4.nc data_5.nc

8    209.3 MiB     52.4 MiB           1       data = xr.open_mfdataset(path)
9                                         
10  82150.1 MiB  81940.8 MiB           1       data = data.load()
11  82101.2 MiB    -49.0 MiB           1       data = data.compute()
12                                         
13  90091.2 MiB   7990.0 MiB           1       data.to_zarr()

``` PS: in this test I just realized I loaded 8 files instead of 5.