issue_comments

No problem ! Thanks everyone for helping me get up to speed :)

Thanks @AndrewWilliams3142

Hello @AndrewWilliams3142! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:

Comment last updated at 2020-05-21 17:16:39 UTC

@keewis thanks for this! I've added what I think is a suitable test for DataArrays, do you think it's also a good idea to have a DataSet test?

chunk is a method of DataArray while rechunk is a method of dask.array.Array

dataarray.chunk("auto").data calls DataArray.chunk and then fetches the underlying data structure (a dask array) while dataarray.data.rechunk("auto") fetches the underlying data structure and then calls dask.array.Array.rechunk.

This could test that dataarray.chunk("auto").data is the same as dataarray.data.rechunk("auto") (or something like that).

@dcherian Thanks for the tip:) Quick question: Is there a reason why you're specifying the .data here? Also I think I'm missing something because I don't get what the difference between .chunk() and .rechunk() would be in this case.

Cheers! I forgot about the tests, will add them this week or next hopefully

Thanks @AndrewWilliams3142 . We should add a test for chunk("auto") . This could test that dataarray.chunk("auto").data is the same as dataarray.data.rechunk("auto") (or something like that).

@dcherian do you have any idea about this mypy Type error? I can't find much (accessible) documentation on how the Union[] is working in this context.

xarray/core/dataset.py:1737: error: Argument 2 to "fromkeys" of "dict" has incompatible type "Union[Number, Mapping[Hashable, Union[None, Number, Tuple[Number, ...]]]]"; expected "Union[None, Number, Tuple[Number, ...]]" xarray/core/dataset.py:1740: error: Item "Number" of "Union[Number, Mapping[Hashable, Union[None, Number, Tuple[Number, ...]]]]" has no attribute "keys"

Edit: thanks to everyone for your help so far!

ah right. this is now rechunking chunked objects to a single chunk when chunks=None. This was not happening previously

Okay so I've traced the error back to the map_blocks() function. I don't fully understand the code for this function in xarray/core/parallel.py, but here's a quick report on the different behaviours.

Normally, when using the make_ds() and make_da() functions in test_dask.py, without any changes to ds.chunk() we have:

```python

def func(obj): ... result = obj + obj.x + 5 * obj.y ... return result ... xr.map_blocks(func, ds).unify_chunks().chunks Frozen(SortedKeysDict({'x': (4, 4, 2), 'y': (5, 5, 5, 5), 'z': (4,)})) func(ds).chunk().unify_chunks().chunks Frozen(SortedKeysDict({'x': (4, 4, 2), 'y': (5, 5, 5, 5), 'z': (4,)})) ```

However, when I use the changes I've made to dataset.py (changing isinstance(chunks, Number) to is_scalar(chunks)), the behaviour becomes:

```python

xr.map_blocks(func, ds).unify_chunks().chunks Frozen(SortedKeysDict({'x': (4, 4, 2), 'y': (5, 5, 5, 5), 'z': (4,)})) func(ds).chunk().unify_chunks().chunks Frozen(SortedKeysDict({'x': (10,), 'y': (20,), 'z': (4,)})) ```

Which means that it now fails the test_map_blocks() call in test_dask.py line 1077.

I've tried to follow through the code and see what is actually happening when this change is made, but I'm out of my depth here. My guess is that is_scalar(chunks) is giving the wrong behaviour when chunks=None ?

Edit: I think that's the problem!
```python

isinstance(None, numbers.Number) False is_scalar(None) True ```

I'll add in something to catch Nones and see if it fixes the error...

then you'll have to run git pull -f once to get the new state

For future reference, github does not work terribly well with rebases (or merges without a merge commit) so it would be good to avoid those.

Edit: ignore the pep8speaks message for now, you didn't change anything in Line 1.

No unpushed commits

if you have any unpushed commits: could you push them now? While they would not be lost, it's way easier to handle them now than after the force-push

Do you mean the master merge? If that's wrong would you be able to fix it for me? My bad, hopefully i'll be able to do it more cleanly in future

there's something wrong with the merge. Are you able to resolve that by yourself, or should I fix it for you?

no, the git pull or git pull origin master (git pull <remote> <branch> in general) should do the trick. "Forking" just means setting up your own repository on github as a mirror of the repository you're forking. After that, git can handle the synchronization between both repositories.

If you need more explanations on git, how to use it and also how it works under the hood, check out the git book.

Edit: the book even has a section on Github

Okay, that makes sense. Though, it seems that I forked the master branch before @kmuehlbauer's commit, which fixed this flake8 issue? So I think I need to make a new fork?

to merge master, what I usually do is: bash $ git checkout master $ git pull # synchronize master with `origin/master` $ git checkout <feature-branch> $ git merge master and if there are merge conflicts (i.e. the merge was interrupted), I follow the advice given by git status

Okay cheers both! I'll have a look at these now.

@keewis sorry I'm still getting used to using this side of Git at the moment, could you clarify what you mean by merge master ? Do you mean merge with my local master?

no need to rebase, simply merge master. Also, you'd then have to remove the checkpoint.

Edit: the failing map_block tests are probably related to your changes (I don't know the code related to dask too well)

I'm not sure why something has changed in these files (I haven't touched them), I also can't work out what the l variable is meant to be doing there. Any tips appreciated!

@AndrewWilliams3142 This is due to flake8 applying new changes from pycodestyle. xarray-devs already dived into this, see https://github.com/pydata/xarray/pull/4057.

You might just need to rebase with current master, to make the error go away.

In my git clone, when I run the flake8 and black . tests, I get the following messages.

(xarray-tests) Andrews-MacBook-Pro-2:xarray andrewwilliams$ black . All done! ✨ 🍰 ✨ 143 files left unchanged. (xarray-tests) Andrews-MacBook-Pro-2:xarray andrewwilliams$ flake8 ./xarray/backends/memory.py:43:32: E741 ambiguous variable name 'l' ./xarray/backends/common.py:244:32: E741 ambiguous variable name 'l' ./xarray/backends/.ipynb_checkpoints/memory-checkpoint.py:43:32: E741 ambiguous variable name 'l'

I'm not sure why something has changed in these files (I haven't touched them), I also can't work out what the l variable is meant to be doing there.

Could this somehow be associated with loads of the checks failing below? Thanks! :)