html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/2534#issuecomment-1114347244,https://api.github.com/repos/pydata/xarray/issues/2534,1114347244,IC_kwDOAMm_X85Ca5bs,5635139,2022-05-01T22:04:43Z,2022-05-01T22:04:43Z,MEMBER,"Sorry these didn't get responses. It's possible that `to_dataframe` is doing some unstacking that's causing huge arrays to be allocated. I don't doubt these examples are real — though the `* 1` example above is very surprising — but we do need reproducible examples to engage.
Please reopen or start a new issue with a reproducible example, thanks.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,376370028
https://github.com/pydata/xarray/issues/2534#issuecomment-1114305976,https://api.github.com/repos/pydata/xarray/issues/2534,1114305976,IC_kwDOAMm_X85CavW4,26384082,2022-05-01T18:37:45Z,2022-05-01T18:37:45Z,NONE,"In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity
If this issue remains relevant, please comment here or remove the `stale` label; otherwise it will be marked as closed automatically
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,376370028
https://github.com/pydata/xarray/issues/2534#issuecomment-613068333,https://api.github.com/repos/pydata/xarray/issues/2534,613068333,MDEyOklzc3VlQ29tbWVudDYxMzA2ODMzMw==,8419421,2020-04-13T19:57:08Z,2020-04-13T19:57:08Z,NONE,"Here is another example:
```
import xarray as xr
ncdata = xr.open_dataset('https://thredds.ucar.edu/thredds/dodsC/nws/metar/ncdecoded/files/Surface_METAR_20200411_0000.nc')
df = ncdata.to_dataframe()
```
Output is:
```
Traceback (most recent call last):
File ""bug.py"", line 4, in
df = ncdata.to_dataframe()
File ""/home/decker/miniconda3/envs/met212/lib/python3.8/site-packages/xarray/core/dataset.py"", line 4399, in to_dataframe
return self._to_dataframe(self.dims)
File ""/home/decker/miniconda3/envs/met212/lib/python3.8/site-packages/xarray/core/dataset.py"", line 4385, in _to_dataframe
data = [
File ""/home/decker/miniconda3/envs/met212/lib/python3.8/site-packages/xarray/core/dataset.py"", line 4386, in
self._variables[k].set_dims(ordered_dims).values.reshape(-1)
MemoryError: Unable to allocate array with shape (117819, 5021) and data type |S64
```
If I'm doing the math right, xarray is trying to allocate roughly 35 GB even though this NetCDF file is only on the order of 50 MB in size.
Output of `xr.show_versions()`
Details
```
INSTALLED VERSIONS
------------------
commit: None
python: 3.8.1 | packaged by conda-forge | (default, Jan 5 2020, 20:58:18)
[GCC 7.3.0]
python-bits: 64
OS: Linux
OS-release: 5.4.31-1-MANJARO
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_US.utf8
LOCALE: en_US.UTF-8
libhdf5: 1.10.5
libnetcdf: 4.7.3
xarray: 0.14.1
pandas: 0.25.3
numpy: 1.17.5
scipy: 1.4.1
netCDF4: 1.5.3
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.0.4.2
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: 3.1.2
cartopy: 0.17.0
seaborn: None
numbagg: None
setuptools: 45.1.0.post20200119
pip: 19.3.1
conda: None
pytest: 5.3.4
IPython: 7.11.1
sphinx: None
```