home / github

Menu
  • Search all tables
  • GraphQL API

issue_comments

Table actions
  • GraphQL API for issue_comments

9 rows where issue = 301031693 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 5

  • shoyer 3
  • max-sixty 2
  • brynpickering 2
  • kmsquire 1
  • FaustinCarter 1

author_association 3

  • MEMBER 5
  • NONE 3
  • CONTRIBUTOR 1

issue 1

  • Removing dimensions from Dataset objects · 9 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
463309270 https://github.com/pydata/xarray/issues/1949#issuecomment-463309270 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDQ2MzMwOTI3MA== kmsquire 223250 2019-02-13T18:20:28Z 2019-02-13T18:20:28Z CONTRIBUTOR

I was looking for a way to drop dimensions, similar to the OP, and found this issue. I created an implementation of Dataset.drop_dims in #2767 which should work.

{
    "total_count": 1,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 1,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693
424025935 https://github.com/pydata/xarray/issues/1949#issuecomment-424025935 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDQyNDAyNTkzNQ== shoyer 1217238 2018-09-24T15:53:49Z 2018-09-24T15:53:49Z MEMBER

So does this mean that the following line in the docs is now false: "If a dimension name is given as an argument to drop, it also drops all variables that use that dimension"

Oops -- yes, that line in the docs / example is broken!

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693
424014244 https://github.com/pydata/xarray/issues/1949#issuecomment-424014244 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDQyNDAxNDI0NA== FaustinCarter 10946756 2018-09-24T15:21:41Z 2018-09-24T15:50:19Z NONE

So does this mean that the following line in the docs is now false: "If a dimension name is given as an argument to drop, it also drops all variables that use that dimension"

This is at http://xarray.pydata.org/en/stable/data-structures.html#dataarray.

It does not seem to work as advertised. Before drop:

<xarray.Dataset> Dimensions: (Frequency: 2000, Index: 1, Power: 1, Temperature: 1) Coordinates: * Power (Power) float64 -117.0 * Temperature (Temperature) float64 1.16 * Index (Index) int64 0 * Frequency (Frequency) float64 3.865e+09 3.865e+09 3.865e+09 3.865e+09 ... Data variables: I (Power, Temperature, Index, Frequency) float64 ... Q (Power, Temperature, Index, Frequency) float64 ... Time (Power, Temperature, Index) datetime64[ns] ...

Then I call ds.drop('Frequency'), expecting that this will kill off the Frequency dimension, coordinate, and the I and Q variables, since they depend on Frequency, but what I get is:

<xarray.Dataset> Dimensions: (Frequency: 2000, Index: 1, Power: 1, Temperature: 1) Coordinates: * Power (Power) float64 -117.0 * Temperature (Temperature) float64 1.16 * Index (Index) int64 0 Dimensions without coordinates: Frequency Data variables: I (Power, Temperature, Index, Frequency) float64 ... Q (Power, Temperature, Index, Frequency) float64 ... Time (Power, Temperature, Index) datetime64[ns] ...

If I then try dropping Frequency again, it complains that there are no variables named 'Frequency'. So probably this issue should include an update to the documentation. Or maybe that should be a new issue.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693
369319591 https://github.com/pydata/xarray/issues/1949#issuecomment-369319591 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDM2OTMxOTU5MQ== shoyer 1217238 2018-02-28T17:38:59Z 2018-02-28T17:38:59Z MEMBER

The second one also provides more scope to remove subsets of data (all corresponding dims, coords, vars) if the dimension becomes superfluous for any reason, whether or not the dimension is empty.

Yes, this was a useful feature that we lost.

Note that in general we try to encourage using methods to create new Datasets rather than modifying existing ones inplace. So it might also make sense to add a drop_dims() method that return a Dataset with the given dimensions removed.

{
    "total_count": 3,
    "+1": 3,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693
369315468 https://github.com/pydata/xarray/issues/1949#issuecomment-369315468 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDM2OTMxNTQ2OA== brynpickering 17178478 2018-02-28T17:27:10Z 2018-02-28T17:27:38Z NONE

The drop technique seems reasonable, if a bit long-winded for the programmatic case (loop over all dimensions, find any that are empty -> loop over all variables, drop any that contain those empty dimensions).

As an addition, if the empty dimension also has an associated empty coordinate then it requires an extra step to get rid of it:

``` python In [21]: test_dataset = xr.Dataset(dict( ...: empty_array=xr.DataArray([], dims='a', coords={'a':[]}), ...: populated_array=xr.DataArray([1], {'b':['1']}, 'b') ...: ))

In [22]: test_dataset Out[22]: <xarray.Dataset> Dimensions: (a: 0, b: 1) Coordinates: * a (a) float64 * b (b) <U1 '1' Data variables: empty_array (a) float64 populated_array (b) int32 1

In [23]: test_dataset.drop('empty_array') Out[23]: <xarray.Dataset> Dimensions: (a: 0, b: 1) Coordinates: * a (a) float64 * b (b) <U1 '1' Data variables: populated_array (b) int32 1

In [24]: del test_dataset['a']

In [25]: test_dataset.drop('empty_array') Out[25]: <xarray.Dataset> Dimensions: (b: 1) Coordinates: * b (b) <U1 '1' Data variables: populated_array (b) int32 1 ```

Fixes seem reasonable, based on how we use xarray over at https://github.com/calliope-project/calliope/. The second one also provides more scope to remove subsets of data (all corresponding dims, coords, vars) if the dimension becomes superfluous for any reason, whether or not the dimension is empty.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693
369294450 https://github.com/pydata/xarray/issues/1949#issuecomment-369294450 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDM2OTI5NDQ1MA== shoyer 1217238 2018-02-28T16:23:40Z 2018-02-28T16:23:40Z MEMBER

If you're OK creating a new Dataset, it works to remove any variables using a dimension, e.g., ``` In [25]: test_dataset = xr.Dataset(dict( ...: empty_array=xr.DataArray([], dims='a'), ...: populated_array=xr.DataArray([1], {'b':['1']}, 'b') ...: )) ...:

In [26]: test_dataset Out[26]: <xarray.Dataset> Dimensions: (a: 0, b: 1) Coordinates: * b (b) <U1 '1' Dimensions without coordinates: a Data variables: empty_array (a) float64 populated_array (b) int64 1

In [27]: test_dataset.drop('empty_array') Out[27]: <xarray.Dataset> Dimensions: (b: 1) Coordinates: * b (b) <U1 '1' Data variables: populated_array (b) int64 1 ```

You're right that this doesn't work to remove dimensions from existing datasets (e.g., with del). But this isn't specific to dimensions of length 0 -- you can't delete any dimensions. At best, you can delete a coordinate variable corresponding to a dimension: ``` In [46]: test_dataset Out[46]: <xarray.Dataset> Dimensions: (a: 0, b: 1) Coordinates: * b (b) <U1 '1' Dimensions without coordinates: a Data variables: empty_array (a) float64 populated_array (b) int64 1

In [47]: del test_dataset['b']

In [48]: test_dataset Out[48]: <xarray.Dataset> Dimensions: (a: 0, b: 1) Dimensions without coordinates: a, b Data variables: empty_array (a) float64 populated_array (b) int64 1

In [49]: del test_dataset['populated_array']

In [50]: test_dataset Out[50]: <xarray.Dataset> Dimensions: (a: 0, b: 1) Dimensions without coordinates: a, b Data variables: empty_array (a) float64 ```

Is there a way to remove a dimension (and possibly then all data variables which depend on it)?

This used to be possible in the xarray data model prior to v0.9.0. del should to delete a dimension corresponding to a coordinate variable and all other associated variables.

When we made coordinates optional, I updated del to only delete data/coordinate variables. This made sense, but meant there is now no way to get rid of dimensions.

I'd like to suggest two possible fixes: 1. Update del dataset[key] to remove all "orphaned" dimensions that don't correspond to a dimension on any variable. This would enforce the invariant "all dimensions on a dataset correspond to a dimension on one of its variables." If there are other places where we violate this invariant those should be fixed, too. 2. Update Dataset.dims to allow deleting elements. Deleting a dimension on a dataset means all associated variables are also deleted.

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693
369267269 https://github.com/pydata/xarray/issues/1949#issuecomment-369267269 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDM2OTI2NzI2OQ== max-sixty 5635139 2018-02-28T15:03:56Z 2018-02-28T15:03:56Z MEMBER

Hmmm, this is harder than I originally expected. I imagine someone will comment with an easy solution, otherwise I'll have another look

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693
369247490 https://github.com/pydata/xarray/issues/1949#issuecomment-369247490 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDM2OTI0NzQ5MA== brynpickering 17178478 2018-02-28T14:00:26Z 2018-02-28T14:00:26Z NONE

I don't think it's actually possible to purge a, hence why I started an issue rather than SO Q. As you can see, squeeze() removes b from the list of dimensions, but not a (as a has a length 0, not 1).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693
369246561 https://github.com/pydata/xarray/issues/1949#issuecomment-369246561 https://api.github.com/repos/pydata/xarray/issues/1949 MDEyOklzc3VlQ29tbWVudDM2OTI0NjU2MQ== max-sixty 5635139 2018-02-28T13:57:18Z 2018-02-28T13:59:43Z MEMBER

I think SO is the best place for user Qs, so the answers can be searchable for future generations.

To respond immediately though, have you tried .squeeze? Or, to confirm, you actually want to remove a rather than b here?

```python In [1]: import xarray as xr

In [2]: test_dataset = xr.Dataset(dict( ...: empty_array=xr.DataArray([], dims='a'), ...: populated_array=xr.DataArray([1], {'b':['1']}, 'b') ...: ))

In [3]: test_dataset Out[3]: <xarray.Dataset> Dimensions: (a: 0, b: 1) Coordinates: * b (b) <U1 '1' Dimensions without coordinates: a Data variables: empty_array (a) float64 populated_array (b) int64 1

In [4]: test_dataset.squeeze() Out[4]: <xarray.Dataset> Dimensions: (a: 0) Coordinates: b <U1 '1' Dimensions without coordinates: a Data variables: empty_array (a) float64 populated_array int64 1 ```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Removing dimensions from Dataset objects 301031693

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 14.181ms · About: xarray-datasette