We've been trying to understand why our code is slow. One part is that we use xarray.Datasets almost like dictionaries for our data. The following code is quite common for us

python import xarray as xr dataset = xr.Dataset() dataset['a'] = 1 dataset['b'] = 2

However, through benchmarks, it became obvious that the merge_core method of xarray was causing alot of slowdowns. main branch:

With this merge request:

```python from tqdm import tqdm import xarray as xr from time import perf_counter import numpy as np

N = 1000

Everybody is lazy loading now, so lets force modules to get instantiated

dummy_dataset = xr.Dataset() dummy_dataset['a'] = 1 dummy_dataset['b'] = 1 del dummy_dataset

time_elapsed = np.zeros(N) dataset = xr.Dataset()

for i in tqdm(range(N)): time_start = perf_counter() dataset[f"var{i}"] = i time_end = perf_counter() time_elapsed[i] = time_end - time_start

%%

from matplotlib import pyplot as plt

plt.plot(np.arange(N), time_elapsed * 1E3, label='Time to add one variable') plt.xlabel("Number of existing variables") plt.ylabel("Time to add a variables (ms)") plt.ylim([0, 50]) plt.grid(True) ```

• [ ] Closes #xxxx
• [ ] Tests added
• [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
• [ ] New functions/methods are listed in api.rst