home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

12 rows where issue = 621078539 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date)

user 7

  • alimanfoo 3
  • shoyer 3
  • keewis 2
  • hmaarrfk 1
  • rabernat 1
  • dcherian 1
  • stale[bot] 1

author_association 3

  • MEMBER 7
  • CONTRIBUTOR 4
  • NONE 1

issue 1

  • Unnamed dimensions · 12 ✖
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
1435715372 https://github.com/pydata/xarray/issues/4079#issuecomment-1435715372 https://api.github.com/repos/pydata/xarray/issues/4079 IC_kwDOAMm_X85Vk0cs keewis 14808389 2023-02-18T16:51:14Z 2023-02-18T16:51:14Z MEMBER

hmm... would np.nan work?

not sure about alignment, but at least obj.sizes would break with multiple dims: while it does not compare as equal, the hash() of np.nan stays constant (same with any other object, I guess).

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
1434959378 https://github.com/pydata/xarray/issues/4079#issuecomment-1434959378 https://api.github.com/repos/pydata/xarray/issues/4079 IC_kwDOAMm_X85Vh74S dcherian 2448579 2023-02-17T17:16:14Z 2023-02-17T17:16:22Z MEMBER

This reminds me of the issue with nan shaped dimensions. Where again we want to avoid automatic alignment (well in that case you can't align).

For this particular issue could we define a xr.UNNAMED that doesn't equal itself (hmm... would np.nan work?). Would that skip alignment checks?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
1434932769 https://github.com/pydata/xarray/issues/4079#issuecomment-1434932769 https://api.github.com/repos/pydata/xarray/issues/4079 IC_kwDOAMm_X85Vh1Yh shoyer 1217238 2023-02-17T17:03:52Z 2023-02-17T17:03:52Z MEMBER

I agree, automatic dimension only ever really made sense for interactive usecases, where a user could see and fix the default names.

It's a little late to change the default now to raising an error instead, but maybe we could add a warning?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
1434805981 https://github.com/pydata/xarray/issues/4079#issuecomment-1434805981 https://api.github.com/repos/pydata/xarray/issues/4079 IC_kwDOAMm_X85VhWbd keewis 14808389 2023-02-17T15:28:07Z 2023-02-17T15:28:07Z MEMBER

I'd probably use itertools.count() or the uuid module to generate globally unique dimension names... something like ```python class UnnamedDimensions: def __init__(self): self.dimension_names = (f"unnamed_dim{number}" for number in itertools.count())

def __call__(self, n):
    return list(itertools.islice(dimension_names, None, n))

unnamed_dimensions = _UnnamedDimensions() or using `uuid` (probably overkill):python def unnamed_dimensions(n): return [uuid.uuid4() for _ in range(n)] you'd use it like this:python d1 = xr.DataArray(data=[1, 2], dims=unnamed_dimensions(1)) d2 = xr.DataArray(data=[[1, 2]], dims=unnamed_dimensions(2)) ``` which would make "unnamed" a bit more explicit.

Edit: that's probably not so different from what you meant with d1_i

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
1434780029 https://github.com/pydata/xarray/issues/4079#issuecomment-1434780029 https://api.github.com/repos/pydata/xarray/issues/4079 IC_kwDOAMm_X85VhQF9 hmaarrfk 90008 2023-02-17T15:08:50Z 2023-02-17T15:08:50Z CONTRIBUTOR

I know it is "stale" but aligning to these "surprise dimensions" creates "late stage" bugs that are hard to pinpoint.

I'm not sure if it is possible to mark these dimensions as "unnamed" and as such, they should be "merged" into new "unnamed" dimensions that the user isn't tracking at this point in time.

Our workaround have included calling these dimensions something related to the datarray d1_i, or simply making small small "arrays" a countable number of scalar variables (d1_min, d1_max) instead of a single array containing two values d1_limits.

```python import xarray as xr d1 = xr.DataArray(data=[1, 2]) assert 'dim_0' in d1.dims d2 = xr.DataArray(data=[1, 2, 3]) assert 'dim_0' in d2.dims

xr.Dataset({'d1': d1, 'd2': d2}) ```

Stack trace ``` --------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[2], line 7 4 d2 = xr.DataArray(data=[1, 2, 3]) 5 assert 'dim_0' in d2.dims ----> 7 xr.Dataset({'d1': d1, 'd2': d2}) File ~/mambaforge/envs/dev/lib/python3.9/site-packages/xarray/core/dataset.py:612, in Dataset.__init__(self, data_vars, coords, attrs) 609 if isinstance(coords, Dataset): 610 coords = coords.variables --> 612 variables, coord_names, dims, indexes, _ = merge_data_and_coords( 613 data_vars, coords, compat="broadcast_equals" 614 ) 616 self._attrs = dict(attrs) if attrs is not None else None 617 self._close = None File ~/mambaforge/envs/dev/lib/python3.9/site-packages/xarray/core/merge.py:564, in merge_data_and_coords(data_vars, coords, compat, join) 562 objects = [data_vars, coords] 563 explicit_coords = coords.keys() --> 564 return merge_core( 565 objects, 566 compat, 567 join, 568 explicit_coords=explicit_coords, 569 indexes=Indexes(indexes, coords), 570 ) File ~/mambaforge/envs/dev/lib/python3.9/site-packages/xarray/core/merge.py:741, in merge_core(objects, compat, join, combine_attrs, priority_arg, explicit_coords, indexes, fill_value) 738 _assert_compat_valid(compat) 740 coerced = coerce_pandas_values(objects) --> 741 aligned = deep_align( 742 coerced, join=join, copy=False, indexes=indexes, fill_value=fill_value 743 ) 744 collected = collect_variables_and_indexes(aligned, indexes=indexes) 745 prioritized = _get_priority_vars_and_indexes(aligned, priority_arg, compat=compat) File ~/mambaforge/envs/dev/lib/python3.9/site-packages/xarray/core/alignment.py:848, in deep_align(objects, join, copy, indexes, exclude, raise_on_invalid, fill_value) 845 else: 846 out.append(variables) --> 848 aligned = align( 849 *targets, 850 join=join, 851 copy=copy, 852 indexes=indexes, 853 exclude=exclude, 854 fill_value=fill_value, 855 ) 857 for position, key, aligned_obj in zip(positions, keys, aligned): 858 if key is no_key: File ~/mambaforge/envs/dev/lib/python3.9/site-packages/xarray/core/alignment.py:785, in align(join, copy, indexes, exclude, fill_value, *objects) 589 """ 590 Given any number of Dataset and/or DataArray objects, returns new 591 objects with aligned indexes and dimension sizes. (...) 775 776 """ 777 aligner = Aligner( 778 objects, 779 join=join, (...) 783 fill_value=fill_value, 784 ) --> 785 aligner.align() 786 return aligner.results File ~/mambaforge/envs/dev/lib/python3.9/site-packages/xarray/core/alignment.py:573, in Aligner.align(self) 571 self.assert_no_index_conflict() 572 self.align_indexes() --> 573 self.assert_unindexed_dim_sizes_equal() 575 if self.join == "override": 576 self.override_indexes() File ~/mambaforge/envs/dev/lib/python3.9/site-packages/xarray/core/alignment.py:472, in Aligner.assert_unindexed_dim_sizes_equal(self) 470 add_err_msg = "" 471 if len(sizes) > 1: --> 472 raise ValueError( 473 f"cannot reindex or align along dimension {dim!r} " 474 f"because of conflicting dimension sizes: {sizes!r}" + add_err_msg 475 ) ValueError: cannot reindex or align along dimension 'dim_0' because of conflicting dimension sizes: {2, 3} ```

cc: @claydugo

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
1112749202 https://github.com/pydata/xarray/issues/4079#issuecomment-1112749202 https://api.github.com/repos/pydata/xarray/issues/4079 IC_kwDOAMm_X85CUzSS stale[bot] 26384082 2022-04-28T23:37:47Z 2022-04-28T23:37:47Z NONE

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
631075010 https://github.com/pydata/xarray/issues/4079#issuecomment-631075010 https://api.github.com/repos/pydata/xarray/issues/4079 MDEyOklzc3VlQ29tbWVudDYzMTA3NTAxMA== alimanfoo 703554 2020-05-19T20:50:26Z 2020-05-19T20:50:51Z CONTRIBUTOR

In the specific example from your notebook, where do the dimensions lengths __variants/BaseCounts_dim1, __variants/MLEAC_dim1 and __variants/MLEAF_dim1 come from?

BaseCounts_dim1 is length 4, so maybe that corresponds to DNA bases ATGC?

In this specific example, I do actually know where these dimension lengths come from. In fact I should've used the shared dimension alt_alleles instead of __variants/MLEAC_dim1 and __variants/MLEAF_dim1. And yes BaseCounts_dim1 does correspond to DNA bases.

But two points.

First, I don't care about these dimensions. The only dimensions I care about and will use are variants, samples and ploidy.

Second, more important, this kind of data can come from a number of different sources, each of which includes a different set of arrays with different names and semantics. While there are some common arrays and naming conventions where I can guess what the dimensions mean, in general I can't know all of those up front and bake them in as special cases.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
630936142 https://github.com/pydata/xarray/issues/4079#issuecomment-630936142 https://api.github.com/repos/pydata/xarray/issues/4079 MDEyOklzc3VlQ29tbWVudDYzMDkzNjE0Mg== shoyer 1217238 2020-05-19T16:33:49Z 2020-05-19T16:33:49Z MEMBER

In the specific example from your notebook, where do the dimensions lengths __variants/BaseCounts_dim1, __variants/MLEAC_dim1 and __variants/MLEAF_dim1 come from?

BaseCounts_dim1 is length 4, so maybe that corresponds to DNA bases ATGC?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
630924754 https://github.com/pydata/xarray/issues/4079#issuecomment-630924754 https://api.github.com/repos/pydata/xarray/issues/4079 MDEyOklzc3VlQ29tbWVudDYzMDkyNDc1NA== alimanfoo 703554 2020-05-19T16:14:27Z 2020-05-19T16:14:27Z CONTRIBUTOR

Thanks @shoyer.

For reference, I'm exploring putting some genome variation data into xarray, here's an initial experiment and discussion here.

In general I will have some arrays where I won't know what some of the dimensions mean, and so cannot give them a meaningful name.

No worries if this is hard, was just wondering if it was supported already.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
630916682 https://github.com/pydata/xarray/issues/4079#issuecomment-630916682 https://api.github.com/repos/pydata/xarray/issues/4079 MDEyOklzc3VlQ29tbWVudDYzMDkxNjY4Mg== shoyer 1217238 2020-05-19T16:00:34Z 2020-05-19T16:00:34Z MEMBER

The main problem is that unnamed dimensions (in NumPy, at least) have different semantics than named dimensions, particular for how dimensions are matched up in broadcasting. For that reason, we really want all dimensions to always be named.

We could possibly add support for unnamed dimensions in the Dataset constructor, but on the other hand it also provides a gentle nudge to chose names.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
630913851 https://github.com/pydata/xarray/issues/4079#issuecomment-630913851 https://api.github.com/repos/pydata/xarray/issues/4079 MDEyOklzc3VlQ29tbWVudDYzMDkxMzg1MQ== alimanfoo 703554 2020-05-19T15:55:54Z 2020-05-19T15:55:54Z CONTRIBUTOR

Thanks so much @rabernat for quick response.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539
630906095 https://github.com/pydata/xarray/issues/4079#issuecomment-630906095 https://api.github.com/repos/pydata/xarray/issues/4079 MDEyOklzc3VlQ29tbWVudDYzMDkwNjA5NQ== rabernat 1197350 2020-05-19T15:42:07Z 2020-05-19T15:42:07Z MEMBER

Is there a way to create arrays and datasets with unnamed dimensions?

No. Xarray's data model requires all dimensions to have a name.

Xarray allows you to create DataArrays without specifying dimensions:

python import xarray as xr import numpy as np da = xr.DataArray(np.zeros((4, 5)))

Xarray will choose dimension names in this case (dim_0, dim_1, etc.)

This doesn't work with creating Datasets, because you have to specify all variables' dimensions in the Dataset constructor.

I imagine that unnamed dimensions would break lots of internal Xarray logic.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  Unnamed dimensions 621078539

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 15.038ms · About: xarray-datasette