id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 176399665,MDU6SXNzdWUxNzYzOTk2NjU=,1001,"""RuntimeError: NetCDF: Not a valid ID"" error when trying to operate on a netCDF file using netCDF4",8268008,closed,0,,,1,2016-09-12T15:03:54Z,2016-09-13T02:33:28Z,2016-09-13T02:33:28Z,NONE,,,,"I have some code that writes out a netCDF file, https://gist.github.com/ceridwen/21f86231e86b72bf18fcc16fcee01548 When I try to do math on this dataset after loading it, though, I get an error: ``` python >>> with xarray.open_dataset('snellman.nc') as dataset: ... print(dataset) ... Dimensions: (event: 91, faction: 20, game: 2643, round: 7) Coordinates: * faction (faction) |S14 b'acolytes' b'alchemists' b'auren' ... * event (event) |S26 b'action:ACT1' b'action:ACT2' b'action:ACT3' ... * game (game) |S32 b'005Stedda' b'006St3dda' b'00s10' b'0106' ... * round (round) int64 0 1 2 3 4 5 6 Data variables: player5 (game) |S17 b'Stedda' b'Muchacho' b'' b'modong' b'' b'' ... players_count (game) int8 5 5 2 5 3 3 4 4 4 4 4 4 5 5 5 5 5 5 5 3 3 4 2 ... date (game) datetime64[ns] 2016-06-07T22:30:47 ... player3 (game) |S19 b'Emoht' b'Xrystofer' b'' b'runrun57' ... player4 (game) |S20 b'Deryl' b'CarpalTunnel' b'' b'Zak_tak' b'' ... player1 (game) |S22 b'ocollard' b'BillyMirmidon' b'00' b'Animax' ... scoring_tiles (game, round) |S6 b'' b'SCORE9' b'SCORE4' b'SCORE2' ... events (game, faction, event, round) float32 nan nan nan nan nan ... player2 (game) |S21 b'Mies730' b'Stedda' b'romance90' ... >>> dataset.mean(dim='round') Traceback (most recent call last): File """", line 1, in File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/common.py"", line 41, in wrapped_func **kwargs) File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/dataset.py"", line 1734, in reduce np.issubdtype(var.dtype, np.number) or File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/variable.py"", line 244, in dtype return self._data.dtype File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/utils.py"", line 385, in dtype return self.array.dtype File ""/home/user/.local/lib/python3.5/site-packages/xarray/conventions.py"", line 436, in dtype return np.dtype('S' + str(self.array.shape[-1])) File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/indexing.py"", line 346, in shape for size, k in zip(self.array.shape, self.key): File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/utils.py"", line 389, in shape return self.array.shape File ""netCDF4/_netCDF4.pyx"", line 3378, in netCDF4._netCDF4.Variable.shape.__get__ (netCDF4/_netCDF4.c:32838) File ""netCDF4/_netCDF4.pyx"", line 3323, in netCDF4._netCDF4.Variable._getdims (netCDF4/_netCDF4.c:31915) RuntimeError: NetCDF: Not a valid ID ``` Note that if I generate the dataset without writing it to a file and loading it, the same operation doesn't cause an error: ``` python >>> dataset.mean(dim='round') Dimensions: (event: 91, faction: 20, game: 2643) Coordinates: * faction (faction) |S14 b'acolytes' b'alchemists' b'auren' ... * event (event) |S26 b'action:ACT1' b'action:ACT2' b'action:ACT3' ... * game (game) object b'005Stedda' b'006St3dda' b'00s10' b'0106' ... Data variables: player5 (game) |S17 b'Stedda' b'Muchacho' b'' b'modong' b'' b'' ... player2 (game) |S21 b'Mies730' b'Stedda' b'romance90' ... player4 (game) |S20 b'Deryl' b'CarpalTunnel' b'' b'Zak_tak' b'' ... player3 (game) |S19 b'Emoht' b'Xrystofer' b'' b'runrun57' ... players_count (game) int8 5 5 2 5 3 3 4 4 4 4 4 4 5 5 5 5 5 5 5 3 3 4 2 ... date (game) datetime64[ns] 2016-06-07T22:30:47 ... player1 (game) |S22 b'ocollard' b'BillyMirmidon' b'00' b'Animax' ... events (game, faction, event) float16 nan nan nan nan nan nan ... ``` This is with xarray (0.8.2) and netCDF4 (1.2.4). It may be a bug in netCDF rather than xarray itself. See also the crash in #1002, which may or may not be related. ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1001/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 176403875,MDU6SXNzdWUxNzY0MDM4NzU=,1002,KeyError when trying to operate on a netCDF file created and read with scipy,8268008,closed,0,,,1,2016-09-12T15:19:52Z,2016-09-13T02:33:14Z,2016-09-13T02:33:14Z,NONE,,,,"This is possibly related to #1001. With the same code and test data, using engine='scipy' in the to_netcdf() and open_dataset() calls, ``` python >>> import xarray >>> with xarray.open_dataset('snellman.nc', engine='scipy') as dataset: ... print(dataset) ... Dimensions: (event: 91, faction: 20, game: 2643, round: 7) Coordinates: * game (game) |S32 b'005Stedda' b'006St3dda' b'00s10' b'0106' ... * event (event) |S26 b'action:ACT1' b'action:ACT2' b'action:ACT3' ... * faction (faction) |S14 b'acolytes' b'alchemists' b'auren' ... * round (round) int64 0 1 2 3 4 5 6 Data variables: player5 (game) |S17 b'Stedda' b'Muchacho' b'' b'modong' b'' b'' ... player2 (game) |S21 b'Mies730' b'Stedda' b'romance90' ... date (game) datetime64[ns] 2016-06-07T22:30:47 ... player3 (game) |S19 b'Emoht' b'Xrystofer' b'' b'runrun57' ... player1 (game) |S22 b'ocollard' b'BillyMirmidon' b'00' b'Animax' ... players_count (game) int8 5 5 2 5 3 3 4 4 4 4 4 4 5 5 5 5 5 5 5 3 3 4 2 ... events (game, faction, event, round) float32 nan nan nan nan nan ... player4 (game) |S20 b'Deryl' b'CarpalTunnel' b'' b'Zak_tak' b'' ... scoring_tiles (game, round) |S6 b'' b'SCORE9' b'SCORE4' b'SCORE2' ... >>> ds.mean(dim='round') Traceback (most recent call last): File """", line 1, in NameError: name 'ds' is not defined >>> dataset.mean(dim='round') Traceback (most recent call last): File """", line 1, in File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/common.py"", line 41, in wrapped_func **kwargs) File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/dataset.py"", line 1734, in reduce np.issubdtype(var.dtype, np.number) or File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/variable.py"", line 244, in dtype return self._data.dtype File ""/home/user/.local/lib/python3.5/site-packages/xarray/core/utils.py"", line 385, in dtype return self.array.dtype File ""/home/user/.local/lib/python3.5/site-packages/xarray/backends/scipy_.py"", line 46, in dtype return np.dtype(self.array.dtype.kind + str(self.array.dtype.itemsize)) File ""/home/user/.local/lib/python3.5/site-packages/xarray/backends/scipy_.py"", line 41, in array return self.netcdf_file.variables[self.variable_name].data KeyError: 'events' ``` As in #1001, the same .mean() call doesn't crash if I'm not trying to load it from a file. This is with xarray (0.8.2) and scipy (0.18.0). ","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/1002/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue