home / github

Menu
  • GraphQL API
  • Search all tables

issues

Table actions
  • GraphQL API for issues

2 rows where comments = 3, type = "issue" and user = 500246 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

state 2

  • closed 1
  • open 1

type 1

  • issue · 2 ✖

repo 1

  • xarray 2
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
283345586 MDU6SXNzdWUyODMzNDU1ODY= 1792 Comparison with masked array yields object-array with nans for masked values gerritholl 500246 open 0     3 2017-12-19T19:37:13Z 2020-10-11T13:34:25Z   CONTRIBUTOR      

Code Sample, a copy-pastable example if possible

``` $ cat mwe.py

!/usr/bin/env python3.6

import xarray import numpy

da = xarray.DataArray(numpy.arange(5)) ma = numpy.ma.masked_array(numpy.arange(5), [True, False, False, False, True]) print(da>ma) $ ./mwe.py <xarray.DataArray (dim_0: 5)> array([nan, False, False, False, nan], dtype=object) Dimensions without coordinates: dim_0 ```

Problem description

A comparison between a DataArray and a masked_array results in an array with dtype object instead of an array with dtype bool. This is problematic, because code should be able to assume that x > y returns something with a bool dtype.

Expected Output

I would expect the masked array to be dropped (which it is) and an array to be returned equivalent to the comparison da>ma.data

<xarray.DataArray (dim_0: 5)> array([False, False, False, False, False], dtype=bool) Dimensions without coordinates: dim_0

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.1.final.0 python-bits: 64 OS: Linux OS-release: 2.6.32-696.6.3.el6.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: en_GB.UTF-8 xarray: 0.10.0+dev12.gf882a58 pandas: 0.21.0 numpy: 1.13.3 scipy: 1.0.0 netCDF4: 1.3.1 h5netcdf: None Nio: None bottleneck: 1.2.1 cyordereddict: None dask: 0.16.0 matplotlib: 2.1.0 cartopy: None seaborn: 0.8.1 setuptools: 38.2.4 pip: 9.0.1 conda: 4.3.16 pytest: 3.1.2 IPython: 6.1.0 sphinx: 1.6.2 None
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1792/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    xarray 13221727 issue
203159853 MDU6SXNzdWUyMDMxNTk4NTM= 1229 opening NetCDF file fails with ValueError when time variable is multidimensional gerritholl 500246 closed 0     3 2017-01-25T16:56:27Z 2017-01-26T05:13:12Z 2017-01-26T05:13:12Z CONTRIBUTOR      

I have a NetCDF file that includes a time field with multiple dimensions. This leads to a failure in xarray.open_dataset, because first_n_items returns an object with shape (1,), but last_item returns an object with shape (1,)*ndim where ndim is the number of dimensions for the time variable. See the illustration below:

``` In [748]: ds = netCDF4.Dataset("test.nc", "w")

In [749]: dim = ds.createDimension("dim", 5)

In [750]: dim2 = ds.createDimension("dim2", 5)

In [751]: time = ds.createVariable("time", "u4", ("dim", "dim2"))

In [752]: time.units = "seconds since 1970-01-01"

In [753]: time.calendar = "gregorian"

In [754]: time[:, :] = arange(25).reshape(5, 5)

In [755]: ds.close()

In [757]: xarray.open_dataset("test.nc")

ValueError Traceback (most recent call last) <ipython-input-757-17ad46b81538> in <module>() ----> 1 xarray.open_dataset("test.nc")

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, group, decode_cf, mask_and_scale, decode_times, concat_characters, decode_coords, engine, chunks, lock, cache, drop_variables) 300 lock = _default_lock(filename_or_obj, engine) 301 with close_on_error(store): --> 302 return maybe_decode_store(store, lock) 303 else: 304 if engine is not None and engine != 'scipy':

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/backends/api.py in maybe_decode_store(store, lock) 221 store, mask_and_scale=mask_and_scale, decode_times=decode_times, 222 concat_characters=concat_characters, decode_coords=decode_coords, --> 223 drop_variables=drop_variables) 224 225 _protect_dataset_variables_inplace(ds, cache)

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/conventions.py in decode_cf(obj, concat_characters, mask_and_scale, decode_times, decode_coords, drop_variables) 947 vars, attrs, coord_names = decode_cf_variables( 948 vars, attrs, concat_characters, mask_and_scale, decode_times, --> 949 decode_coords, drop_variables=drop_variables) 950 ds = Dataset(vars, attrs=attrs) 951 ds = ds.set_coords(coord_names.union(extra_coords).intersection(vars))

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/conventions.py in decode_cf_variables(variables, attributes, concat_characters, mask_and_scale, decode_times, decode_coords, drop_variables) 882 new_vars[k] = decode_cf_variable( 883 v, concat_characters=concat, mask_and_scale=mask_and_scale, --> 884 decode_times=decode_times) 885 if decode_coords: 886 var_attrs = new_vars[k].attrs

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/conventions.py in decode_cf_variable(var, concat_characters, mask_and_scale, decode_times, decode_endianness) 819 units = pop_to(attributes, encoding, 'units') 820 calendar = pop_to(attributes, encoding, 'calendar') --> 821 data = DecodedCFDatetimeArray(data, units, calendar) 822 elif attributes['units'] in TIME_UNITS: 823 # timedelta

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/conventions.py in init(self, array, units, calendar) 384 # Dataset.repr when users try to view their lazily decoded array. 385 example_value = np.concatenate([first_n_items(array, 1) or [0], --> 386 last_item(array) or [0]]) 387 388 try:

ValueError: all the input arrays must have same number of dimensions ```

Closer look in the debugger:

``` In [758]: %debug xarray.open_dataset("test.nc") NOTE: Enter 'c' at the ipdb> prompt to continue execution.

<string>(1)<module>()

ipdb> break /dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/conventions.py:385 Breakpoint 1 at /dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/conventions.py:385 ipdb> cont

/dev/shm/gerrit/venv/stable-3.5/lib/python3.5/site-packages/xarray/conventions.py(385)init() 383 # successfully. Otherwise, tracebacks end up swallowed by 384 # Dataset.repr when users try to view their lazily decoded array. 1-> 385 example_value = np.concatenate([first_n_items(array, 1) or [0], 386 last_item(array) or [0]]) 387

ipdb> p first_n_items(array, 1).shape (1,) ipdb> p last_item(array).shape (1, 1) ```

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1229/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);
Powered by Datasette · Queries took 27.199ms · About: xarray-datasette