home / github

Menu
  • Search all tables
  • GraphQL API

issues

Table actions
  • GraphQL API for issues

3 rows where state = "open", type = "issue" and user = 13906519 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

type 1

  • issue · 3 ✖

state 1

  • open · 3 ✖

repo 1

  • xarray 3
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
809332917 MDU6SXNzdWU4MDkzMzI5MTc= 4914 Record processing steps into history attribute with context manager cwerner 13906519 open 0     4 2021-02-16T13:55:05Z 2022-05-23T13:28:48Z   NONE      

I often want to record an entry into history of my netcdf file/ xarray. While one can always add it manually, i.e.

python ds.attrs["history"] = ds.attrs["history"] + "\n" + "message"

I was wondering if there's a better way... In a first attempt I tried using a context manager for this. Not sure if there are other approaches? Would that be something useful for xarray core? What are other people using for this?

Demo:

```python import datetime import xarray as xr

class XrHistory():

def __init__(self, array, message, timestamp=True):
    self._array = array
    self._message = message
    self._timestamp = timestamp

def __enter__(self):
    if 'history' not in self._array.attrs:
        self._array.attrs['history'] = ""

    if self._message != self._array.attrs['history'].split('\n')[-1]:
        ts = f"{datetime.datetime.now().strftime('%a %b %d %H:%M:%S %Y')}: " if self._timestamp else ""

        self._array.attrs['history'] += f"\n{ts}{self._message}"
        self._message = None
    return self._array

def __exit__(self, exc_type,exc_value, exc_traceback):
    pass

ds is any xarray dataset...

with XrHistory(ds, "normalise data") as ds: ds["array_one"] = (ds.array_one - ds.array_one.mean(dim='time')) / ds.array_one.std(dim='time')

with XrHistory(ds, "subset data") as ds: ds = ds.sel(x=slice(10, 20), y=slice(10,20))

...

```

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4914/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    xarray 13221727 issue
616025171 MDU6SXNzdWU2MTYwMjUxNzE= 4052 Opendap access problem when subsetting via latitude and longitude... cwerner 13906519 open 0     1 2020-05-11T16:44:02Z 2022-04-29T00:37:51Z   NONE      

I am trying to access a subset of a netcdf files hosted on a THREDDS server. I can inspect the metadata, but I cannot subset the file via lat and lon slices. A download via the http link provided on the page works...

MCVE Code Sample

```python import xarray as xr

this works

test1 = xr.open_dataset(URL) display(test1)

this also works

test2 = xr.open_dataset(URL).sel(lat=slice(30,40)) display(test2)

this also works

test3 = xr.open_dataset(URL).sel(lon=slice(100,110)) display(test3)

this fails

test4 = xr.open_dataset(URL).sel(lat=slice(30,40), lon=slice(100,110)) display(test4)

```

Problem Description

Error: ``` ~/.pyenv/versions/miniconda3-latest/envs/datascience/lib/python3.7/site-packages/xarray/backends/common.py in robust_getitem(array, key, catch, max_retries, initial_delay) 52 for n in range(max_retries + 1): 53 try: ---> 54 return array[key] 55 except catch: 56 if n == max_retries:

netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.getitem()

netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable._get()

netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()

RuntimeError: NetCDF: Access failure ```

I also tried to use the pydap engine, but I'm not sure if this tells me something about the problem or if I use this option incorrectly...

```python

URL = "https://thredds.daac.ornl.gov/thredds/dodsC/ornldaac/1247/T_CLAY.nc4" test1 = xr.open_dataset(URL, engine='pydap') test1

```

result: UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128)

Versions

``` INSTALLED VERSIONS


commit: None python: 3.7.4 (default, Aug 13 2019, 15:17:50) [Clang 4.0.1 (tags/RELEASE_401/final)] python-bits: 64 OS: Darwin OS-release: 19.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1

xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.4.2 pydap: installed h5netcdf: None h5py: None Nio: None zarr: 2.4.0 cftime: 1.0.4.2 nc_time_axis: None PseudoNetCDF: None rasterio: 1.0.21 cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.11.0 matplotlib: 3.1.3 cartopy: 0.17.0 seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200210 pip: 20.0.2 conda: None pytest: None IPython: 7.12.0 sphinx: None ```

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4052/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    xarray 13221727 issue
650549352 MDU6SXNzdWU2NTA1NDkzNTI= 4197 Provide a "shrink" command to remove bounding nan/ whitespace of DataArray cwerner 13906519 open 0     7 2020-07-03T11:55:05Z 2022-04-09T01:22:31Z   NONE      

I'm currently trying to come up with an elegant solution to remove extra whitespace/ nan-values along the edges of a 2D DataArray. I'm working with geographic data and search for an automatic way to shrink the extend to valid data only. Think a map of the EU, but remove all cols/ rows of the array (starting from the edges) that only contain nan.

Describe the solution you'd like A shrink command that removes all nan rows/ cols at the edges of a DataArray.

Describe alternatives you've considered I currently do this with NumPy operating on the raw data and creating a new DataArray afterwards

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4197/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);
Powered by Datasette · Queries took 3765.619ms · About: xarray-datasette