id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1981799811,I_kwDOAMm_X852H92D,8423,Support remote string paths for `h5netcdf` engine,11656932,open,0,,,4,2023-11-07T16:52:18Z,2023-11-09T07:24:45Z,,CONTRIBUTOR,,,,"### Is your feature request related to a problem? Currently the `h5netcdf` engine supports opening remote files, but only already open file-like objects (e.g. `s3fs.open(...)`), not string paths like `s3://...`. There are situations where I'd like to use string paths instead of open file-like objets - Opening files can sometimes be slow (xref https://github.com/fsspec/s3fs/issues/816) - When using `parallel=True` for opening lots of files, serializing open file-like objects back and forth from a remote cluster can be slow - Some systems (e.g. NASA Earthdata) only hand out credentials that are valid when run in the same region as the data. Being able to use `parallel=True` + `storage_options` would be convenient/performant in that case. ### Describe the solution you'd like It would be nice if I could do something like the following: ```python ds = xr.open_mfdataset( files, # A bunch of files like `s3://bucket/file` engine=""h5netcdf"", ... parallel=True, storage_options={...}, # fsspec-compatible options ) ``` and have my files opened prior to handing off to `h5netcdf`. `storage_options` is already supported for Zarr, so hopefully extending to `h5netcdf` feels natural. ### Describe alternatives you've considered _No response_ ### Additional context _No response_","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/8423/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 1562851680,I_kwDOAMm_X85dJzlg,7490,`pre-commit` hooks broken,11656932,closed,0,,,1,2023-01-30T17:19:49Z,2023-01-30T18:15:50Z,2023-01-30T18:15:50Z,CONTRIBUTOR,,,,"### What happened? When I attempt to run `pre-commit` locally I get the following error: ``` ❯ pre-commit run --all-files [INFO] Installing environment for https://github.com/PyCQA/isort. [INFO] Once installed this environment will be reused. [INFO] This may take a few minutes... An unexpected error has occurred: CalledProcessError: command: ('/Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/bin/python', '-mpip', 'install', '.') return code: 1 expected return code: 0 stdout: Processing /Users/james/.cache/pre-commit/repo_eax0t7q Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing wheel metadata: started Preparing wheel metadata: finished with status 'error' stderr: DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default. pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555. ERROR: Command errored out with exit status 1: command: /Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/bin/python /Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /var/folders/k_/lx1rdvqn253gd1wrcx__5frm0000gn/T/tmphq1i398_ cwd: /private/var/folders/k_/lx1rdvqn253gd1wrcx__5frm0000gn/T/pip-req-build-4ywk4y5l Complete output (14 lines): Traceback (most recent call last): File ""/Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py"", line 280, in main() File ""/Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py"", line 263, in main json_out['return_val'] = hook(**hook_input['kwargs']) File ""/Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py"", line 133, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) File ""/private/var/folders/k_/lx1rdvqn253gd1wrcx__5frm0000gn/T/pip-build-env-bvdexj4q/overlay/lib/python3.9/site-packages/poetry/core/masonry/api.py"", line 40, in prepare_metadata_for_build_wheel poetry = Factory().create_poetry(Path(""."").resolve(), with_groups=False) File ""/private/var/folders/k_/lx1rdvqn253gd1wrcx__5frm0000gn/T/pip-build-env-bvdexj4q/overlay/lib/python3.9/site-packages/poetry/core/factory.py"", line 57, in create_poetry raise RuntimeError(""The Poetry configuration is invalid:\n"" + message) RuntimeError: The Poetry configuration is invalid: - [extras.pipfile_deprecated_finder.2] 'pip-shims<=0.3.4' does not match '^[a-zA-Z-_.0-9]+$' ---------------------------------------- WARNING: Discarding file:///Users/james/.cache/pre-commit/repo_eax0t7q. Command errored out with exit status 1: /Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/bin/python /Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /var/folders/k_/lx1rdvqn253gd1wrcx__5frm0000gn/T/tmphq1i398_ Check the logs for full command output. ERROR: Command errored out with exit status 1: /Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/bin/python /Users/james/.cache/pre-commit/repo_eax0t7q/py_env-python3.9/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /var/folders/k_/lx1rdvqn253gd1wrcx__5frm0000gn/T/tmphq1i398_ Check the logs for full command output. Check the log at /Users/james/.cache/pre-commit/pre-commit.log ``` It also looks like we're seeing the same thing in CI https://results.pre-commit.ci/run/github/13221727/1675098797.6-zAnTMbQXC74StmKz5PFw ### What did you expect to happen? _No response_ ### Minimal Complete Verifiable Example _No response_ ### MVCE confirmation - [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. - [ ] Complete example — the example is self-contained, including all data and the text of any traceback. - [ ] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result. - [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. ### Relevant log output _No response_ ### Anything else we need to know? _No response_ ### Environment
``` INSTALLED VERSIONS ------------------ commit: 12f6d2449b478afbcf442a39bc3f0048b1fa8c29 python: 3.9.6 | packaged by conda-forge | (default, Jul 11 2021, 03:36:15) [Clang 11.1.0 ] python-bits: 64 OS: Darwin OS-release: 22.2.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.19.0 pandas: 1.3.2 numpy: 1.21.2 scipy: 1.7.1 netCDF4: 1.5.6 pydap: installed h5netcdf: 0.11.0 h5py: 3.3.0 Nio: None zarr: 2.8.3 cftime: 1.5.0 nc_time_axis: 1.3.1 PseudoNetCDF: installed rasterio: 1.2.6 cfgrib: 0.9.9.0 iris: 3.0.4 bottleneck: 1.3.2 dask: 2023.1.1+1.ga8327a35c distributed: 2023.1.1 matplotlib: 3.4.3 cartopy: 0.19.0.post1 seaborn: 0.11.2 numbagg: 0.2.1 fsspec: 2021.06.1 cupy: None pint: 0.17 sparse: 0.13.0 flox: None numpy_groupies: None setuptools: 57.4.0 pip: 21.2.4 conda: None pytest: 6.2.4 mypy: None IPython: None sphinx: None ```
","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7490/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue 960674159,MDU6SXNzdWU5NjA2NzQxNTk=,5675,Use of `force_to_top` for `isort`,11656932,closed,0,,,1,2021-08-04T16:01:09Z,2021-08-21T20:15:17Z,2021-08-21T20:15:17Z,CONTRIBUTOR,,,,"In https://github.com/dask/distributed/pull/5170, @fjetter pointed out that `distributed` was incorrectly using `force_to_top` in our `isort` configuration (we should have been using `float_to_top` instead). Just a minor point, but since `xarray`s `isort` configuration influenced what we did over in `dask` and `distributed`, I thought it was worth letting the Xarray devs know about it https://github.com/pydata/xarray/blob/8f5b4a185b304ab68723c390b1ad88e57f9a60d6/setup.cfg#L164","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/5675/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue