html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/3416#issuecomment-544312894,https://api.github.com/repos/pydata/xarray/issues/3416,544312894,MDEyOklzc3VlQ29tbWVudDU0NDMxMjg5NA==,56707780,2019-10-21T01:04:48Z,2019-10-21T01:04:48Z,NONE,"Got it, thanks everyone. I'll open this issue there. We'll try and work on getting our NetCDF4 compatibility issues addressed to avoid this space issue, as we are working with large imaging datasets.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,509285415
https://github.com/pydata/xarray/issues/3416#issuecomment-544292045,https://api.github.com/repos/pydata/xarray/issues/3416,544292045,MDEyOklzc3VlQ29tbWVudDU0NDI5MjA0NQ==,6213168,2019-10-20T20:59:40Z,2019-10-20T21:00:07Z,MEMBER,"The error message is clearly improvable, however this ticket belongs to the scipy board. @angelolab please reopen it there. All ok with closing this?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,509285415
https://github.com/pydata/xarray/issues/3416#issuecomment-544291094,https://api.github.com/repos/pydata/xarray/issues/3416,544291094,MDEyOklzc3VlQ29tbWVudDU0NDI5MTA5NA==,1217238,2019-10-20T20:47:59Z,2019-10-20T20:47:59Z,MEMBER,SciPy's netCDF module has an additional limitation that would likely be limiting for writing large files: it currently writes netCDF files by loading the entire file into memory at once. Ideally it would support incrementally writing data.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,509285415
https://github.com/pydata/xarray/issues/3416#issuecomment-544290771,https://api.github.com/repos/pydata/xarray/issues/3416,544290771,MDEyOklzc3VlQ29tbWVudDU0NDI5MDc3MQ==,1217238,2019-10-20T20:44:38Z,2019-10-20T20:44:38Z,MEMBER,"This array is 7.5 GB in size, which unfortunately is too big to be stored as a single record in the netCDF3 file format, even the 64-bit version: 
https://www.unidata.ucar.edu/software/netcdf/docs/file_format_specifications.html#offset_format_spec

We could potentially support writing these variables to disk with scipy, but it would require refactoring xarray's scipy netCDF backend to write record variables (with indeterminate size), which we currently don't use.

In the meanwhile, the best work-around is to use the netCDF4 file format, either via the netCDF4 Python package or h5netcdf.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,509285415
https://github.com/pydata/xarray/issues/3416#issuecomment-544183597,https://api.github.com/repos/pydata/xarray/issues/3416,544183597,MDEyOklzc3VlQ29tbWVudDU0NDE4MzU5Nw==,13770365,2019-10-19T18:19:08Z,2019-10-19T18:19:33Z,NONE,"Yes, you're right @max-sixty, seems like the actual IO is happening within scipy. Do you have any suggestions for how I might troubleshoot this further? Could I make a similar example that uses only the scipy netcdf functionality somehow to drill down into where the memory error is coming from?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,509285415
https://github.com/pydata/xarray/issues/3416#issuecomment-544179019,https://api.github.com/repos/pydata/xarray/issues/3416,544179019,MDEyOklzc3VlQ29tbWVudDU0NDE3OTAxOQ==,5635139,2019-10-19T17:44:09Z,2019-10-19T17:44:09Z,MEMBER,"Thanks for the issue @angelolab 

Are you sure this is an xarray error? It looks like the error is being raised from scipy","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,509285415