id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 277538485,MDU6SXNzdWUyNzc1Mzg0ODU=,1745,open_mfdataset() memory error in v0.10,22665917,closed,0,,,24,2017-11-28T21:08:23Z,2019-01-13T01:51:43Z,2019-01-13T01:51:43Z,NONE,,,,"#### Code Sample ```python import xarray ncfiles = '/example/path/to/wrf/netcdfs/*' dropvars = ['list', 'of', 'many', 'vars', 'to', 'drop'] dset = xarray.open_mfdataset(ncfiles, drop_variables=dropvars, concat_dim='Time', autoclose=True, decode_cf=False) ``` #### Problem description I am trying to load 73 model (WRF) output files using ``open_mfdataset()``. (Thus, 'Time' is a new dimension). Each netcdf has dimensions ``{'x' : 405, 'y' : 282, 'z': 37}`` and roughly 20 variables (excluding the other ~20 in ``dropvars``). When I run the above code with **v0.9.6**, it completes in roughly 7 seconds. But with **v0.10**, it crashes with the following error: ``*** Error in `~/anaconda3/bin/python': corrupted size vs. prev_size: 0x0000560e9b6ca7b0 ***`` which, as I understand, means I'm exceeding my memory allocation. Any thoughts on what could be the source of this issue? #### Output of ``xr.show_versions()``
INSTALLED VERSIONS ------------------ commit: None python: 3.6.3.final.0 python-bits: 64 OS: Linux OS-release: 4.9.0-3-amd64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: C LANG: C LOCALE: None.None xarray: 0.10.0 pandas: 0.20.3 numpy: 1.13.1 scipy: 0.19.1 netCDF4: 1.2.4 h5netcdf: 0.5.0 Nio: None bottleneck: 1.2.1 cyordereddict: None dask: 0.16.0 matplotlib: 2.0.2 cartopy: None seaborn: 0.8.0 setuptools: 27.2.0 pip: 9.0.1 conda: 4.3.29 pytest: 3.1.3 IPython: 6.1.0 sphinx: 1.6.2
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1745/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 374070147,MDU6SXNzdWUzNzQwNzAxNDc=,2512,to_netcdf() fails because of datetime encoding,22665917,closed,0,,,2,2018-10-25T18:17:47Z,2018-10-27T16:34:54Z,2018-10-27T16:34:54Z,NONE,,,,"#### Simple example: ```python import numpy as np from datetime import datetime, timedelta import xarray # ""time"" coordinate dt = datetime(1999, 1, 1) dts = np.array([dt + timedelta(days=x) for x in range(10)]) coords = {'time': dts} # simple float data data = np.arange(10) vrbls = {'foo': (('time',), data)} # create the Dataset ds = xarray.Dataset(vrbls, coords) # encode the time coordinate units = 'days since 1900-01-01' ds.time.encoding['units'] = units # write to netcdf ds.to_netcdf('test.nc') ``` #### Problem description When I run the above, I get the following error when executing the last line: **`ValueError: unsupported dtype for netCDF4 variable: datetime64[ns]`** The documentation indicates that datetime and datetime64 objects are both supported by xarray and should write to netcdf just fine when supplied ""units"" for encoding (this code fails with or without the encoding lines). Any Idea when is going wrong here? #### Output of ``xr.show_versions()``
INSTALLED VERSIONS ------------------ commit: None python: 3.6.2.final.0 python-bits: 64 OS: Linux OS-release: 4.9.0-8-amd64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 xarray: 0.10.9 pandas: 0.20.3 numpy: 1.13.1 scipy: 0.19.1 netCDF4: 1.4.2 h5netcdf: 0.5.0 h5py: 2.8.0 Nio: None zarr: None cftime: 1.0.1 PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: 0.16.0 distributed: 1.20.1 matplotlib: 2.1.0 cartopy: None seaborn: 0.8.0 setuptools: 27.2.0 pip: 9.0.1 conda: 4.5.11 pytest: 3.1.3 IPython: 6.1.0 sphinx: 1.6.2
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/2512/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue