html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/3795#issuecomment-591709959,https://api.github.com/repos/pydata/xarray/issues/3795,591709959,MDEyOklzc3VlQ29tbWVudDU5MTcwOTk1OQ==,14808389,2020-02-27T00:13:12Z,2020-02-27T00:13:12Z,MEMBER,"Is there a reason you need to have exactly one chunk? The chunks are used to reduce the memory needed when computing the final results and you should only need to care about them if you are trying to optimize the speed / memory footprint of your operations.
If you really need to have exactly one chunk (and you have enough memory), it might be better to use `ds.load()` or `ds.compute()` directly after the call to `open_mfdataset` and continue without `dask`.
I'm not really experienced with the backends / `open_mfdataset` / `dask`, though, so I'd appreciate input from someone with more experience.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418
https://github.com/pydata/xarray/issues/3795#issuecomment-590375940,https://api.github.com/repos/pydata/xarray/issues/3795,590375940,MDEyOklzc3VlQ29tbWVudDU5MDM3NTk0MA==,34353851,2020-02-24T15:21:42Z,2020-02-24T15:26:57Z,NONE,"df1=xarray.open_mfdataset plus parallel=True
df1=df1.rename({'__xarray_dataarray_variable__':'v'})
The chunksize changes to 365 and the dataset creates inside a Datarray of 365, not the global size which is 14610. [xarray.Dataset('u':df1,'v'df2)]
df2 same operation
Pseudo Solution for me.
xarray.Dataset({'u':df1.u.chunk(14610),'v':df2.v.chunk(14610)},coords={'time':time_Index,'latitude':latitude_Index,'longitude':longitude_Index,'level':level_Index})","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418
https://github.com/pydata/xarray/issues/3795#issuecomment-590369650,https://api.github.com/repos/pydata/xarray/issues/3795,590369650,MDEyOklzc3VlQ29tbWVudDU5MDM2OTY1MA==,14808389,2020-02-24T15:10:40Z,2020-02-24T15:10:40Z,MEMBER,"I don't think `xarray.open_dataset` has a `parallel` option so that should probably be `xarray.open_mfdataset`?
Sorry, I should have been clear on what I requested. You don't have to dive into the `xarray` code, I just wanted you to show how you use `xarray` so I can understand and reproduce your issue. We typically point to http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports for more explanations and the reasoning.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418
https://github.com/pydata/xarray/issues/3795#issuecomment-590351872,https://api.github.com/repos/pydata/xarray/issues/3795,590351872,MDEyOklzc3VlQ29tbWVudDU5MDM1MTg3Mg==,34353851,2020-02-24T14:39:11Z,2020-02-24T15:06:16Z,NONE,"
backends\api.py
DATAARRAY_NAME = ""__xarray_dataarray_name__""
DATAARRAY_VARIABLE = ""__xarray_dataarray_variable__""
The name is auto when i open the file ,xarrray.open_dataset( parallel=True)
core\dataarray.py
def rename(
self,
new_name_or_name_dict: Union[Hashable, Mapping[Hashable, Hashable]] = None,
**names: Hashable,
) -> ""DataArray"":
if names or utils.is_dict_like(new_name_or_name_dict):
new_name_or_name_dict = cast(
Mapping[Hashable, Hashable], new_name_or_name_dict
)
name_dict = either_dict_or_kwargs(new_name_or_name_dict, names, ""rename"")
**dataset = self.**_to_temp_dataset()**.rename(name_dict)
return self._from_temp_dataset(dataset)**
else:
new_name_or_name_dict = cast(Hashable, new_name_or_name_dict)
return self._replace(name=new_name_or_name_dict)
I think an operation changing the previous chunksize or something with xarrray.open_dataset parallel=True, (core\parallel.py) because the chunksize changes to 365 based on days of the year
Sorry i cannot help more.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418
https://github.com/pydata/xarray/issues/3795#issuecomment-590316603,https://api.github.com/repos/pydata/xarray/issues/3795,590316603,MDEyOklzc3VlQ29tbWVudDU5MDMxNjYwMw==,14808389,2020-02-24T13:21:08Z,2020-02-24T13:21:08Z,MEMBER,"that looks fine to me.
> The strange is the DataArray chunksize changes after i modify the name 'xarray_dataarray_variable' to use another to the Dataset.
modify how? I can't find anything about `xarray_dataarray_variable`, so the code would be helpful.
> After i modify again the chunksize to 365 to avoid the Memory Error
again, showing how you modify (in code) would help a lot
To really be able to help you and to exclude misunderstandings, I need a bit more information. To be clear, what I would like to know is:
* the `repr` of the original dataset (is that the one from https://github.com/pydata/xarray/issues/3795#issuecomment-590294555?)
* code for the operation you want to do (a description or pseudo-code is fine, too)
* the `repr` of the data structure you get (or want to get)
* and optionally, if there is something wrong with the result, a description of what you would have expected","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418
https://github.com/pydata/xarray/issues/3795#issuecomment-590297372,https://api.github.com/repos/pydata/xarray/issues/3795,590297372,MDEyOklzc3VlQ29tbWVudDU5MDI5NzM3Mg==,34353851,2020-02-24T12:30:34Z,2020-02-24T12:30:34Z,NONE,"After i modify again the chunksize to 365 to avoid the Memory Error
<xarray.Dataset>
Dimensions: (latitude: 68, level: 47, longitude: 81, time: 14610)
Data variables:
u (time, level, latitude, longitude) float32 dask.array<shape=(14610, 47, 68, 81), chunksize=(365, 47, 68, 81)>
v (time, level, latitude, longitude) float32 dask.array<shape=(14610, 47, 68, 81), chunksize=(365, 47, 68, 81)>","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418
https://github.com/pydata/xarray/issues/3795#issuecomment-590294555,https://api.github.com/repos/pydata/xarray/issues/3795,590294555,MDEyOklzc3VlQ29tbWVudDU5MDI5NDU1NQ==,34353851,2020-02-24T12:21:39Z,2020-02-24T12:21:39Z,NONE,"The strange is the DataArray chunksize changes after i modify the name '__xarray_dataarray_variable__' to use another to the Dataset.
<xarray.Dataset>
Dimensions: (latitude: 68, level: 47, longitude: 81, time: 14610)
Data variables:
u (time, level, latitude, longitude) float32 dask.array<shape=(14610, 47, 68, 81), chunksize=(14610, 47, 68, 81)>
v (time, level, latitude, longitude) float32 dask.array<shape=(14610, 47, 68, 81), chunksize=(14610, 47, 68, 81)>","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418
https://github.com/pydata/xarray/issues/3795#issuecomment-590290745,https://api.github.com/repos/pydata/xarray/issues/3795,590290745,MDEyOklzc3VlQ29tbWVudDU5MDI5MDc0NQ==,14808389,2020-02-24T12:10:03Z,2020-02-24T12:10:03Z,MEMBER,"Could you show a code example demonstrating how you created your dataset / dataarrays and what you are trying to get? For example, something like this:
```python
In [1]: import xarray as xr
...: import dask.array as da
In [2]: a = da.zeros((14610, 47, 68, 81), chunks=(365, 47, 68, 81))
...: b = da.zeros((10950, 47, 68, 81), chunks=(365, 47, 68, 81))
...: ds = xr.Dataset({""a"": ((""u"", ""x"", ""y"", ""z""), a), ""b"": ((""v"", ""x"", ""y"", ""z""), b)})
...: ds
Out[2]:
<xarray.Dataset>
Dimensions: (u: 14610, v: 10950, x: 47, y: 68, z: 81)
Dimensions without coordinates: u, v, x, y, z
Data variables:
a (u, x, y, z) float64 dask.array<chunksize=(365, 47, 68, 81), meta=np.ndarray>
b (v, x, y, z) float64 dask.array<chunksize=(365, 47, 68, 81), meta=np.ndarray>
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,569806418