home / github

Menu
  • GraphQL API
  • Search all tables

issues

Table actions
  • GraphQL API for issues

2 rows where "closed_at" is on date 2021-01-04 and user = 14371165 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

type 2

  • issue 1
  • pull 1

state 1

  • closed 2

repo 1

  • xarray 2
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
775322346 MDU6SXNzdWU3NzUzMjIzNDY= 4736 Limit number of data variables shown in repr Illviljan 14371165 closed 0     2 2020-12-28T10:15:26Z 2021-01-04T02:13:52Z 2021-01-04T02:13:52Z MEMBER      

What happened: xarray feels very unresponsive when using datasets with >2000 data variables because it has to print all the 2000 variables everytime you print something to console.

What you expected to happen: xarray should limit the number of variables printed to console. Maximum maybe 25? Same idea probably apply to dimensions, coordinates and attributes as well,

pandas only shows 2 for reference, the first and last variables.

Minimal Complete Verifiable Example:

```python import numpy as np import xarray as xr

a = np.arange(0, 2000) b = np.core.defchararray.add("long_variable_name", a.astype(str)) data_vars = dict() for v in b: data_vars[v] = xr.DataArray( name=v, data=[3, 4], dims=["time"], coords=dict(time=[0, 1]) ) ds = xr.Dataset(data_vars)

Everything above feels fast. Printing to console however takes about 13 seconds for me:

print(ds) ```

Anything else we need to know?: Out of scope brainstorming: Though printing 2000 variables is probably madness for most people it is kind of nice to show all variables because you sometimes want to know what happened to a few other variables as well. Is there already an easy and fast way to create subgroup of the dataset, so we don' have to rely on the dataset printing everything to the console everytime?

Environment:

Output of <tt>xr.show_versions()</tt> xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 libhdf5: 1.10.4 libnetcdf: None xarray: 0.16.2 pandas: 1.1.5 numpy: 1.17.5 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: None nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2020.12.0 distributed: 2020.12.0 matplotlib: 3.3.2 cartopy: None seaborn: 0.11.1 numbagg: None pint: None setuptools: 51.0.0.post20201207 pip: 20.3.3 conda: 4.9.2 pytest: 6.2.1 IPython: 7.19.0 sphinx: 3.4.0
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4736/reactions",
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed xarray 13221727 issue
777526340 MDExOlB1bGxSZXF1ZXN0NTQ3Nzk5MDk2 4750 Limit number of data rows shown in repr Illviljan 14371165 closed 0     8 2021-01-02T21:14:50Z 2021-01-04T02:13:52Z 2021-01-04T02:13:52Z MEMBER   0 pydata/xarray/pulls/4750
  • [x] Closes #4736
  • [x] Tests added
  • [x] Passes isort . && black . && mypy . && flake8
  • [ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
  • [ ] New functions/methods are listed in api.rst

Test example: python a = np.arange(0, 2000) b = np.core.defchararray.add("long_variable_name", a.astype(str)) c = np.arange(0, 30) d = np.core.defchararray.add("attr_", c.astype(str)) e = {k: 2 for k in d} coords = dict(time=da.array([0, 1])) data_vars = dict() for v in b: data_vars[v] = xr.DataArray( name=v, data=da.array([3, 4]), dims=["time"], coords=coords, ) ds0 = xr.Dataset(data_vars) ds0.attrs = e

Looks like this with 24 max rows of interesting data: python print(ds0) Out[15]: <xarray.Dataset> Dimensions: (time: 2) Coordinates: * time (time) int32 0 1 Data variables: long_variable_name0 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name2 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name3 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name4 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name5 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name6 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name7 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name8 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name9 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name10 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name11 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> ... ... long_variable_name1988 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1989 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1990 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1991 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1992 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1993 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1994 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1995 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1996 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1997 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1998 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1999 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> Attributes: attr_0: 2 attr_1: 2 attr_2: 2 attr_3: 2 attr_4: 2 attr_5: 2 attr_6: 2 attr_7: 2 attr_8: 2 attr_9: 2 attr_10: 2 attr_11: 2 ... ... attr_18: 2 attr_19: 2 attr_20: 2 attr_21: 2 attr_22: 2 attr_23: 2 attr_24: 2 attr_25: 2 attr_26: 2 attr_27: 2 attr_28: 2 attr_29: 2 With 20 rows of interesting data: python xr.set_options(display_max_rows=20) ds0 Out[26]: <xarray.Dataset> Dimensions: (time: 2) Coordinates: * time (time) int32 0 1 Data variables: long_variable_name0 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name2 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name3 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name4 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name5 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name6 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name7 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name8 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name9 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> ... long_variable_name1990 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1991 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1992 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1993 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1994 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1995 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1996 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1997 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1998 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1999 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> Attributes: attr_0: 2 attr_1: 2 attr_2: 2 attr_3: 2 attr_4: 2 attr_5: 2 attr_6: 2 attr_7: 2 attr_8: 2 attr_9: 2 ... attr_20: 2 attr_21: 2 attr_22: 2 attr_23: 2 attr_24: 2 attr_25: 2 attr_26: 2 attr_27: 2 attr_28: 2 attr_29: 2

With 16 rows of interesting data: python xr.set_options(display_max_rows=16) ds0 Out[28]: <xarray.Dataset> Dimensions: (time: 2) Coordinates: * time (time) int32 0 1 Data variables: long_variable_name0 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name2 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name3 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name4 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name5 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name6 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name7 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> ... long_variable_name1992 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1993 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1994 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1995 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1996 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1997 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1998 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1999 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> Attributes: attr_0: 2 attr_1: 2 attr_2: 2 attr_3: 2 attr_4: 2 attr_5: 2 attr_6: 2 attr_7: 2 ... attr_22: 2 attr_23: 2 attr_24: 2 attr_25: 2 attr_26: 2 attr_27: 2 attr_28: 2 attr_29: 2

With 12 rows of interesting data: ```python xr.set_options(display_max_rows=12) print(ds0) Out[79]: <xarray.Dataset> Dimensions: (time: 2) Coordinates: * time (time) int32 0 1 Data variables: long_variable_name0 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name2 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name3 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name4 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name5 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> ... long_variable_name1994 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1995 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1996 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1997 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1998 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> long_variable_name1999 (time) int32 dask.array<chunksize=(2,), meta=np.ndarray> Attributes: attr_0: 2 attr_1: 2 attr_2: 2 attr_3: 2 attr_4: 2 attr_5: 2 ... attr_24: 2 attr_25: 2 attr_26: 2 attr_27: 2 attr_28: 2 attr_29: 2

```

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4750/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    xarray 13221727 pull

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);
Powered by Datasette · Queries took 429.443ms · About: xarray-datasette