html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/873#issuecomment-248415522,https://api.github.com/repos/pydata/xarray/issues/873,248415522,MDEyOklzc3VlQ29tbWVudDI0ODQxNTUyMg==,1217238,2016-09-20T19:55:55Z,2016-09-20T19:55:55Z,MEMBER,"Great, glad that worked!
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,158958801
https://github.com/pydata/xarray/issues/873#issuecomment-248409634,https://api.github.com/repos/pydata/xarray/issues/873,248409634,MDEyOklzc3VlQ29tbWVudDI0ODQwOTYzNA==,1328158,2016-09-20T19:37:07Z,2016-09-20T19:37:07Z,NONE,"Thanks for this clarification, Stephan. Apparently I didn't read the API
documentation closely enough, as I was assuming that the function is
applied to the underlying ndarray rather than to all data variables of a
Dataset object. Now that I've taken the approach you suggested I'm cooking
with gas, and it's very encouraging. I really appreciate your help.
--James
On Tue, Sep 20, 2016 at 11:54 AM, Stephan Hoyer notifications@github.com
wrote:
> GroupBy is working as intended here. ds.groupby('something').apply(func)
> calls func on objects of the same type as ds. If you group a Dataset,
> each time you apply to a Dataset, too.
>
> You can certainly still use np.convolve, but you'll need to manually
> apply it to numpy arrays extracted from a Dataset and then rebuild another
> Dataset or DataArray.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> https://github.com/pydata/xarray/issues/873#issuecomment-248345053, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/ABREHkSB-DvW5OD3DHuYxYzP7l7yfjL2ks5qsAGcgaJpZM4IwE4g
> .
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,158958801
https://github.com/pydata/xarray/issues/873#issuecomment-248345053,https://api.github.com/repos/pydata/xarray/issues/873,248345053,MDEyOklzc3VlQ29tbWVudDI0ODM0NTA1Mw==,1217238,2016-09-20T15:54:02Z,2016-09-20T15:54:02Z,MEMBER,"GroupBy is working as intended here. `ds.groupby('something').apply(func)` calls `func` on objects of the same type as `ds`. If you group a `Dataset`, each time you apply to a `Dataset`, too.
You can certainly still use `np.convolve`, but you'll need to manually apply it to numpy arrays extracted from a Dataset and then rebuild another Dataset or DataArray.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,158958801
https://github.com/pydata/xarray/issues/873#issuecomment-248216388,https://api.github.com/repos/pydata/xarray/issues/873,248216388,MDEyOklzc3VlQ29tbWVudDI0ODIxNjM4OA==,1328158,2016-09-20T06:42:53Z,2016-09-20T06:42:53Z,NONE,"Thanks, Stephan. My code uses numpy.convolve() in several key places, so if that function is a deal breaker for using xarray then I'll hold off until that's fixed. In the meantime if there's anything else I can do to help you work this out then please let me know.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,158958801
https://github.com/pydata/xarray/issues/873#issuecomment-242951236,https://api.github.com/repos/pydata/xarray/issues/873,242951236,MDEyOklzc3VlQ29tbWVudDI0Mjk1MTIzNg==,1217238,2016-08-28T01:53:14Z,2016-08-28T01:53:14Z,MEMBER,"The first issue is an xarray bug. See #989 for a fix.
The work around is not to provide an encoded dtype if the variable already has the dtype you want.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,158958801
https://github.com/pydata/xarray/issues/873#issuecomment-242950771,https://api.github.com/repos/pydata/xarray/issues/873,242950771,MDEyOklzc3VlQ29tbWVudDI0Mjk1MDc3MQ==,1217238,2016-08-28T01:37:44Z,2016-08-28T01:37:44Z,MEMBER,"The second issue is that `compute_function` is being called on an `xarray.Dataset`, so `np.convolve(data, data * 2)` is attempting to convolve two xarray Datasets. NumPy gives a pretty unhelpful error message. (If only we had static typing in Python...)
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,158958801
https://github.com/pydata/xarray/issues/873#issuecomment-242535724,https://api.github.com/repos/pydata/xarray/issues/873,242535724,MDEyOklzc3VlQ29tbWVudDI0MjUzNTcyNA==,1328158,2016-08-25T20:48:45Z,2016-08-25T20:48:45Z,NONE,"Thanks, Stephan. In general things appear to be working much more as
expected now, probably (hopefully) this is just an edge case/nuance that
won't be too difficult for you guys to address. If so and if I don't run
across any other issues then my code will be dramatically simplified by
leveraging xarray rather than writing code to enable shared memory objects
for the multiprocessing side of things (my assumption being that you guys
have done a better job of that than I can).
A gist with example code and a smallish data file attached to the comment
is here: https://gist.github.com/monocongo/e8e883c2355f7a92bb0b9d24db5407a8
Please let me know if I can do anything else to help you help me. Godspeed!
--James
On Tue, Aug 23, 2016 at 12:42 AM, Stephan Hoyer notifications@github.com
wrote:
> Could you please share a data file and/or code which I can run to
> reproduce each of these issues?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> https://github.com/pydata/xarray/issues/873#issuecomment-241625354, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/ABREHhoGvwv30D2Qk858lHB-U5oWtRQnks5qinpDgaJpZM4IwE4g
> .
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,158958801
https://github.com/pydata/xarray/issues/873#issuecomment-241625354,https://api.github.com/repos/pydata/xarray/issues/873,241625354,MDEyOklzc3VlQ29tbWVudDI0MTYyNTM1NA==,1217238,2016-08-23T04:42:41Z,2016-08-23T04:42:41Z,MEMBER,"Could you please share a data file and/or code which I can run to reproduce each of these issues?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,158958801
https://github.com/pydata/xarray/issues/873#issuecomment-241540585,https://api.github.com/repos/pydata/xarray/issues/873,241540585,MDEyOklzc3VlQ29tbWVudDI0MTU0MDU4NQ==,1328158,2016-08-22T20:32:20Z,2016-08-22T20:32:20Z,NONE,"I get the following error now when I try to run the gist code referenced in the original message above:
```
$ python -u xarray_gist.py /dev/shm/nclimgrid_prcp_reduced.nc nclimgrid_prcp_doubled.nc
Traceback (most recent call last):
File ""xarray_gist.py"", line 45, in
encoding = {variable_name: {'_FillValue': np.nan, 'dtype': 'float32'}})
File ""/home/james.adams/anaconda3/lib/python3.5/site-packages/xarray/core/dataset.py"", line 782, in to_netcdf
engine=engine, encoding=encoding)
File ""/home/james.adams/anaconda3/lib/python3.5/site-packages/xarray/backends/api.py"", line 354, in to_netcdf
dataset.dump_to_store(store, sync=sync, encoding=encoding)
File ""/home/james.adams/anaconda3/lib/python3.5/site-packages/xarray/core/dataset.py"", line 728, in dump_to_store
store.store(variables, attrs, check_encoding)
File ""/home/james.adams/anaconda3/lib/python3.5/site-packages/xarray/backends/common.py"", line 234, in store
check_encoding_set)
File ""/home/james.adams/anaconda3/lib/python3.5/site-packages/xarray/backends/common.py"", line 209, in store
self.set_variables(variables, check_encoding_set)
File ""/home/james.adams/anaconda3/lib/python3.5/site-packages/xarray/backends/common.py"", line 219, in set_variables
target, source = self.prepare_variable(name, v, check)
File ""/home/james.adams/anaconda3/lib/python3.5/site-packages/xarray/backends/netCDF4_.py"", line 266, in prepare_variable
raise_on_invalid=check_encoding)
File ""/home/james.adams/anaconda3/lib/python3.5/site-packages/xarray/backends/netCDF4_.py"", line 167, in _extract_nc4_encoding
' %r' % (backend, invalid))
ValueError: unexpected encoding parameters for 'netCDF4' backend: ['dtype']
```
Additionally I see the following errors when I run some other code which uses the same dataset.groupby().apply() technique (the trouble appears to show up within numpy.convolve()):
```
Traceback (most recent call last):
File ""C:\home\git\indices\src\main\python\indices\spi_gamma_xarray.py"", line 46, in
dataset = dataset.groupby('grid_cells').apply(function_to_be_applied)
File ""C:\Anaconda3\lib\site-packages\xarray\core\groupby.py"", line 567, in apply
combined = self._concat(applied)
File ""C:\Anaconda3\lib\site-packages\xarray\core\groupby.py"", line 572, in _concat
applied_example, applied = peek_at(applied)
File ""C:\Anaconda3\lib\site-packages\xarray\core\utils.py"", line 90, in peek_at
peek = next(gen)
File ""C:\Anaconda3\lib\site-packages\xarray\core\groupby.py"", line 566, in
applied = (func(ds, **kwargs) for ds in self._iter_grouped())
File ""C:\home\git\indices\src\main\python\indices\spi_gamma_xarray.py"", line 27, in function_to_be_applied
valid_max)
File ""C:\Anaconda3\lib\site-packages\numpy\core\numeric.py"", line 1005, in convolve
return multiarray.correlate(a, v[::-1], mode)
TypeError: Cannot cast array data from dtype('float64') to dtype('
dataset = dataset.groupby('grid_cells').apply(function_to_be_applied)
File ""C:\Anaconda3\lib\site-packages\xarray\core\groupby.py"", line 567, in apply
combined = self._concat(applied)
File ""C:\Anaconda3\lib\site-packages\xarray\core\groupby.py"", line 572, in _concat
applied_example, applied = peek_at(applied)
File ""C:\Anaconda3\lib\site-packages\xarray\core\utils.py"", line 90, in peek_at
peek = next(gen)
File ""C:\Anaconda3\lib\site-packages\xarray\core\groupby.py"", line 566, in
applied = (func(ds, **kwargs) for ds in self._iter_grouped())
File ""C:\home\git\indices\src\main\python\indices\spi_gamma_xarray.py"", line 27, in function_to_be_applied
valid_max)
File ""C:\Anaconda3\lib\site-packages\numpy\core\numeric.py"", line 1005, in convolve
return multiarray.correlate(a, v[::-1], mode)
TypeError: Cannot cast array data from dtype('float64') to dtype('