html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/pull/6402#issuecomment-1094118500,https://api.github.com/repos/pydata/xarray/issues/6402,1094118500,IC_kwDOAMm_X85BNuxk,38358698,2022-04-09T20:19:02Z,2022-04-09T20:19:02Z,CONTRIBUTOR,"> @stanwest how's that?
That looks great to me.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1177669703
https://github.com/pydata/xarray/pull/6402#issuecomment-1093300764,https://api.github.com/repos/pydata/xarray/issues/6402,1093300764,IC_kwDOAMm_X85BKnIc,38358698,2022-04-08T19:51:24Z,2022-04-08T19:51:24Z,CONTRIBUTOR,"> @stanwest Can you suggest how to fix this merge conflict please?
Sure. I recommend the following before the `return` statement in `xarray.core.dataset._get_chunk`:
```python
# Warn where requested chunks break preferred chunks, provided that the variable
# contains data.
if var.size:
for dim, size, chunk_sizes in zip(dims, shape, chunk_shape):
try:
preferred_chunk_sizes = preferred_chunks[dim]
except KeyError:
continue
# Determine the stop indices of the preferred chunks, but omit the last stop
# (equal to the dim size). In particular, assume that when a sequence
# expresses the preferred chunks, the sequence sums to the size.
preferred_stops = (
range(preferred_chunk_sizes, size, preferred_chunk_sizes)
if isinstance(preferred_chunk_sizes, Number)
else itertools.accumulate(preferred_chunk_sizes[:-1])
)
# Gather any stop indices of the specified chunks that are not a stop index
# of a preferred chunk. Again, omit the last stop, assuming that it equals
# the dim size.
breaks = set(itertools.accumulate(chunk_sizes[:-1])).difference(
preferred_stops
)
if breaks:
warnings.warn(
""The specified Dask chunks separate the stored chunks along ""
f'dimension ""{dim}"" starting at index {min(breaks)}. This could '
""degrade performance. Instead, consider rechunking after loading.""
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1177669703
https://github.com/pydata/xarray/pull/6334#issuecomment-1093061133,https://api.github.com/repos/pydata/xarray/issues/6334,1093061133,IC_kwDOAMm_X85BJsoN,38358698,2022-04-08T16:23:01Z,2022-04-08T16:23:01Z,CONTRIBUTOR,"For my understanding and curiosity, what are the perceived benefits of a separate function to warn about splitting the preferred chunks? It seemed better to me to avoid the overhead of the function call and the unnecessary internal interface, particularly considering that the function was private, had only one caller, and was so closely related to its caller.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1160073438
https://github.com/pydata/xarray/pull/6334#issuecomment-1091797026,https://api.github.com/repos/pydata/xarray/issues/6334,1091797026,IC_kwDOAMm_X85BE4Ai,38358698,2022-04-07T14:15:13Z,2022-04-07T14:15:13Z,CONTRIBUTOR,"> Thank you for this fix! It looks good to me.
Thanks for the review and approval. Is this PR ready to trade the ""needs review"" label for the ""plan to merge"" label?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1160073438
https://github.com/pydata/xarray/pull/6305#issuecomment-1054400787,https://api.github.com/repos/pydata/xarray/issues/6305,1054400787,IC_kwDOAMm_X84-2OET,38358698,2022-02-28T15:56:34Z,2022-02-28T15:56:34Z,CONTRIBUTOR,Thanks for accepting the fix!,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1150484906
https://github.com/pydata/xarray/pull/6305#issuecomment-1050997589,https://api.github.com/repos/pydata/xarray/issues/6305,1050997589,IC_kwDOAMm_X84-pPNV,38358698,2022-02-25T16:22:13Z,2022-02-25T16:22:13Z,CONTRIBUTOR,"For the sake of cross-references, #6249 originated the test modified here.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1150484906
https://github.com/pydata/xarray/pull/6237#issuecomment-1035162721,https://api.github.com/repos/pydata/xarray/issues/6237,1035162721,IC_kwDOAMm_X849s1Rh,38358698,2022-02-10T16:56:22Z,2022-02-10T16:56:22Z,CONTRIBUTOR,Okay. I've committed the changes.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,1124431593
https://github.com/pydata/xarray/pull/6237#issuecomment-1034136761,https://api.github.com/repos/pydata/xarray/issues/6237,1034136761,IC_kwDOAMm_X849o6y5,38358698,2022-02-09T19:53:45Z,2022-02-09T19:53:45Z,CONTRIBUTOR,"> Thanks for your PR - we are definitely interested to make this work on windows.
👍
> Instead of adding `:okexcept:` could we explicitly close the offending files?
It seems that we can, although closing all of the references was non-trivial in ""user-guide/io.rst"". Please see the diffs below for the two files with the most extensive changes. Within each document, I've tried to delete the files as early as possible to keep that code close to the last use of the file.
""user-guide/dask.rst""
```diff
@@ -55,6 +55,8 @@ argument to :py:func:`~xarray.open_dataset` or using the
.. ipython:: python
:suppress:
+ import os
+
import numpy as np
import pandas as pd
import xarray as xr
@@ -129,6 +131,11 @@ will return a ``dask.delayed`` object that can be computed later.
with ProgressBar():
results = delayed_obj.compute()
+.. ipython:: python
+ :suppress:
+
+ os.remove(""manipulated-example-data.nc"") # Was not opened.
+
.. note::
When using Dask's distributed scheduler to write NETCDF4 files,
@@ -147,14 +154,6 @@ A dataset can also be converted to a Dask DataFrame using :py:meth:`~xarray.Data
Dask DataFrames do not support multi-indexes so the coordinate variables from the dataset are included as columns in the Dask DataFrame.
-.. ipython:: python
- :okexcept:
- :suppress:
-
- import os
-
- os.remove(""example-data.nc"")
- os.remove(""manipulated-example-data.nc"")
Using Dask with xarray
----------------------
@@ -211,7 +210,7 @@ Dask arrays using the :py:meth:`~xarray.Dataset.persist` method:
.. ipython:: python
- ds = ds.persist()
+ persisted = ds.persist()
:py:meth:`~xarray.Dataset.persist` is particularly useful when using a
distributed cluster because the data will be loaded into distributed memory
@@ -233,11 +232,6 @@ chunk size depends both on your data and on the operations you want to perform.
With xarray, both converting data to a Dask arrays and converting the chunk
sizes of Dask arrays is done with the :py:meth:`~xarray.Dataset.chunk` method:
-.. ipython:: python
- :suppress:
-
- ds = ds.chunk({""time"": 10})
-
.. ipython:: python
rechunked = ds.chunk({""latitude"": 100, ""longitude"": 100})
@@ -509,6 +503,11 @@ Notice that the 0-shaped sizes were not printed to screen. Since ``template`` ha
expected = ds + 10 + 10
mapped.identical(expected)
+.. ipython:: python
+ :suppress:
+
+ ds.close() # Closes ""example-data.nc"".
+ os.remove(""example-data.nc"")
.. tip::
```
Above, I've removed the line `ds = ds.chunk({""time"": 10})`, because the call to open the dataset already specified that chunking.
""user-guide/io.rst""
```diff
@@ -11,6 +11,8 @@ format (recommended).
.. ipython:: python
:suppress:
+ import os
+
import numpy as np
import pandas as pd
import xarray as xr
@@ -84,6 +86,13 @@ We can load netCDF files to create a new Dataset using
ds_disk = xr.open_dataset(""saved_on_disk.nc"")
ds_disk
+.. ipython:: python
+ :suppress:
+
+ # Close ""saved_on_disk.nc"", but retain the file until after closing or deleting other
+ # datasets that will refer to it.
+ ds_disk.close()
+
Similarly, a DataArray can be saved to disk using the
:py:meth:`DataArray.to_netcdf` method, and loaded
from disk using the :py:func:`open_dataarray` function. As netCDF files
@@ -204,11 +213,6 @@ You can view this encoding information (among others) in the
Note that all operations that manipulate variables other than indexing
will remove encoding information.
-.. ipython:: python
- :suppress:
-
- ds_disk.close()
-
.. _combining multiple files:
@@ -484,14 +488,13 @@ and currently raises a warning unless ``invalid_netcdf=True`` is set:
da.to_netcdf(""complex.nc"", engine=""h5netcdf"", invalid_netcdf=True)
# Reading it back
- xr.open_dataarray(""complex.nc"", engine=""h5netcdf"")
+ reopened = xr.open_dataarray(""complex.nc"", engine=""h5netcdf"")
+ reopened
.. ipython:: python
- :okexcept:
:suppress:
- import os
-
+ reopened.close()
os.remove(""complex.nc"")
.. warning::
@@ -724,17 +727,19 @@ To export just the dataset schema without the data itself, use the
ds.to_dict(data=False)
-This can be useful for generating indices of dataset contents to expose to
-search indices or other automated data discovery tools.
-
.. ipython:: python
- :okexcept:
:suppress:
- import os
-
+ # We're now done with the dataset named `ds`. Although the `with` statement closed
+ # the dataset, displaying the unpickled pickle of `ds` re-opened ""saved_on_disk.nc"".
+ # However, `ds` (rather than the unpickled dataset) refers to the open file. Delete
+ # `ds` to close the file.
+ del ds
os.remove(""saved_on_disk.nc"")
+This can be useful for generating indices of dataset contents to expose to
+search indices or other automated data discovery tools.
+
.. _io.rasterio:
Rasterio
```
""internals/zarr-encoding-spec.rst""
```diff
@@ -63,3 +63,9 @@ re-open it directly with Zarr:
print(os.listdir(""rasm.zarr""))
print(zgroup.tree())
dict(zgroup[""Tair""].attrs)
+
+.. ipython:: python
+ :suppress:
+
+ import shutil
+ shutil.rmtree(""rasm.zarr"")
```