home / github

Menu
  • GraphQL API
  • Search all tables

issue_comments

Table actions
  • GraphQL API for issue_comments

8 rows where issue = 778069594 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: reactions, created_at (date), updated_at (date)

user 3

  • mathause 4
  • keewis 3
  • dcherian 1

issue 1

  • WIP: testing.assert_* check dtype · 8 ✖

author_association 1

  • MEMBER 8
id html_url issue_url node_id user created_at updated_at ▲ author_association body reactions performed_via_github_app issue
945825537 https://github.com/pydata/xarray/pull/4760#issuecomment-945825537 https://api.github.com/repos/pydata/xarray/issues/4760 IC_kwDOAMm_X844YCcB mathause 10194086 2021-10-18T14:21:56Z 2021-10-18T14:21:56Z MEMBER

I closed the PR by accident but I think I'll keep it closed, unless there is interest to get this in. Testing dtypes could be an interesting option, however,

  • there are O(1000) tests that do not preserve dtype - so this would be an enormous cleanup required if we ever seriously wanted to do this
  • some elements cannot preserve dtypes (str types in pandas index (not sure what the index refactor does here) and empty str arrays)
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WIP: testing.assert_* check dtype 778069594
774112777 https://github.com/pydata/xarray/pull/4760#issuecomment-774112777 https://api.github.com/repos/pydata/xarray/issues/4760 MDEyOklzc3VlQ29tbWVudDc3NDExMjc3Nw== mathause 10194086 2021-02-05T15:44:58Z 2021-02-05T16:14:22Z MEMBER

True, but for the problem that the diff repr is not shown I think it makes sense to go for this, e.g. the following works fine:

```python def test_x():

data_array1 = xr.DataArray(data=1 * unit_registry.m)
data_array2 = xr.DataArray(data=2 * unit_registry.m)

with warnings.catch_warnings():
    warnings.simplefilter("always")

    xr.testing.assert_allclose(data_array1, data_array2)

```

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WIP: testing.assert_* check dtype 778069594
774106999 https://github.com/pydata/xarray/pull/4760#issuecomment-774106999 https://api.github.com/repos/pydata/xarray/issues/4760 MDEyOklzc3VlQ29tbWVudDc3NDEwNjk5OQ== keewis 14808389 2021-02-05T15:35:18Z 2021-02-05T16:00:17Z MEMBER

to really fix this we need some way to say "I really want to convert to a numpy.ndarray" (unfortunately, np.asarray seems to be commonly used as "ensure that this is a numpy.ndarray"). That's the same issue as with cupy.Array, which will raise in __array__.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WIP: testing.assert_* check dtype 778069594
774115528 https://github.com/pydata/xarray/pull/4760#issuecomment-774115528 https://api.github.com/repos/pydata/xarray/issues/4760 MDEyOklzc3VlQ29tbWVudDc3NDExNTUyOA== keewis 14808389 2021-02-05T15:49:32Z 2021-02-05T15:59:52Z MEMBER

yes, I agree that it makes sense to set warnings to "always" in the assert_* functions.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WIP: testing.assert_* check dtype 778069594
774101639 https://github.com/pydata/xarray/pull/4760#issuecomment-774101639 https://api.github.com/repos/pydata/xarray/issues/4760 MDEyOklzc3VlQ29tbWVudDc3NDEwMTYzOQ== mathause 10194086 2021-02-05T15:27:27Z 2021-02-05T15:27:27Z MEMBER

Thanks! That drove me nuts but I finally figured out the problem:

https://github.com/pydata/xarray/blob/5735e163bea43ec9bc3c2e640fbf25a1d4a9d0c0/xarray/tests/test_units.py#L29

promotes the warning to an error so diff_array_repr fails (because the units are stripped...) and that is what we see. That also fails on master - run the following in with pytest:

```python import pint import pytest import xarray as xr

pytestmark = [ pytest.mark.filterwarnings("error::pint.UnitStrippedWarning"), ]

unit_registry = pint.UnitRegistry(force_ndarray=True)

def test_y():

data_array1 = xr.DataArray(data=1 * unit_registry.m)
data_array2 = xr.DataArray(data=2 * unit_registry.m)

xr.testing.assert_allclose(data_array1, data_array2)

```

@keewis do you have an idea how to fix this? One idea is to supress warnings in diff_array_repr and diff_dataset_repr. Do we need to strip the units before sending them to these functions?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WIP: testing.assert_* check dtype 778069594
774024142 https://github.com/pydata/xarray/pull/4760#issuecomment-774024142 https://api.github.com/repos/pydata/xarray/issues/4760 MDEyOklzc3VlQ29tbWVudDc3NDAyNDE0Mg== keewis 14808389 2021-02-05T13:10:27Z 2021-02-05T14:14:20Z MEMBER

the difference is that the unit registry has force_ndarray_like enabled. This is important because a pint.Quantity wrapping a python scalar is not a duck array, so the unit gets stripped.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WIP: testing.assert_* check dtype 778069594
774001075 https://github.com/pydata/xarray/pull/4760#issuecomment-774001075 https://api.github.com/repos/pydata/xarray/issues/4760 MDEyOklzc3VlQ29tbWVudDc3NDAwMTA3NQ== mathause 10194086 2021-02-05T12:22:41Z 2021-02-05T12:22:41Z MEMBER

@keewis The following test is giving me trouble: pytest xarray/tests/test_units.py::test_align_dataarray[int64-10-data-compatible_unit]

https://github.com/pydata/xarray/blob/5735e163bea43ec9bc3c2e640fbf25a1d4a9d0c0/xarray/tests/test_units.py#L461

I now try to reproduce what it does but I cannot even manage that... I think the following is what the test does but it fails with DimensionalityError: Cannot convert from 'dimensionless' to 'meter' - can you see what I do wrong/ different from the test?

```python import numpy as np import pint import xarray as xr

unit_registry = pint.UnitRegistry()

dtype = np.int64

data_unit1 = unit_registry.m data_unit2 = unit_registry.mm

array1 = np.linspace(0, 10, 2 * 5).reshape(2, 5).astype(dtype) array2 = np.linspace(0, 8, 2 * 5).reshape(2, 5).astype(dtype)

x = np.arange(2) # * dim_unit1 y1 = np.arange(5) # * dim_unit1 y2 = np.arange(2, 7) # * dim_unit2

u1 = np.array([3, 5, 7, 8, 9]) # * coord_unit1 u2 = np.array([7, 8, 9, 11, 13]) # * coord_unit2

coords1 = {"x": x, "y": y1} coords2 = {"x": x, "y": y2}

data_array1 = xr.DataArray(data=array1 * data_unit1, coords=coords1, dims=("x", "y")) data_array2 = xr.DataArray(data=array2 * data_unit2, coords=coords2, dims=("x", "y"))

fill_value = 10

actual_a, actual_b = xr.align(data_array1, data_array2, join="outer", fill_value=fill_value * data_unit1)

```


The underlying problem is that when I run this test with check_dtype=True it raises an error (which is fine) but instead of a nice diff_repr it elevates the UnitStrippedWarning to an error and I have not the slightest clue why...

```python-traceback > assert_allclose(expected_b, actual_b) /home/mathause/code/xarray/xarray/tests/test_units.py:542: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/mathause/code/xarray/xarray/tests/__init__.py:208: in assert_allclose xarray.testing.assert_allclose(a, b, check_dtype=check_dtype, **kwargs) /home/mathause/code/xarray/xarray/testing.py:178: in assert_allclose assert allclose, formatting.diff_array_repr( /home/mathause/code/xarray/xarray/core/formatting.py:667: in diff_array_repr if not equiv(a, b, check_dtype=check_dtype): /home/mathause/code/xarray/xarray/testing.py:40: in _data_allclose_or_equiv return duck_array_ops.allclose_or_equiv( /home/mathause/code/xarray/xarray/core/duck_array_ops.py:249: in allclose_or_equiv arr1 = asarray(arr1) /home/mathause/code/xarray/xarray/core/duck_array_ops.py:184: in asarray return data if is_duck_array(data) else xp.asarray(data) /home/mathause/conda/envs/xarray_dev/lib/python3.8/site-packages/numpy/core/_asarray.py:102: in asarray return array(a, dtype, copy=False, order=order) /home/mathause/code/xarray/xarray/core/common.py:139: in __array__ return np.asarray(self.values, dtype=dtype) /home/mathause/code/xarray/xarray/core/dataarray.py:634: in values return self.variable.values /home/mathause/code/xarray/xarray/core/variable.py:544: in values return _as_array_or_item(self._data) /home/mathause/code/xarray/xarray/core/variable.py:276: in _as_array_or_item data = np.asarray(data) /home/mathause/conda/envs/xarray_dev/lib/python3.8/site-packages/numpy/core/_asarray.py:102: in asarray return array(a, dtype, copy=False, order=order) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Quantity([[10. 10. 0. 0. 1. 2. 3.] [10. 10. 4. 5. 6. 7. 8.]], 'millimeter')>, t = None def __array__(self, t=None): > warnings.warn( "The unit of the quantity is stripped when downcasting to ndarray.", UnitStrippedWarning, stacklevel=2, ) E pint.errors.UnitStrippedWarning: The unit of the quantity is stripped when downcasting to ndarray. /home/mathause/conda/envs/xarray_dev/lib/python3.8/site-packages/pint/quantity.py:1683: UnitStrippedWarning ```
{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WIP: testing.assert_* check dtype 778069594
754142883 https://github.com/pydata/xarray/pull/4760#issuecomment-754142883 https://api.github.com/repos/pydata/xarray/issues/4760 MDEyOklzc3VlQ29tbWVudDc1NDE0Mjg4Mw== dcherian 2448579 2021-01-04T18:36:52Z 2021-01-04T18:36:52Z MEMBER

sometimes dask can get the dtype wrong before the compute (see below)

I suspect this is only for object arrays since you need not get back an object array? Maybe check in lazy_array_equiv only if all dtypes are not object

{
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  WIP: testing.assert_* check dtype 778069594

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issue_comments] (
   [html_url] TEXT,
   [issue_url] TEXT,
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [created_at] TEXT,
   [updated_at] TEXT,
   [author_association] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [issue] INTEGER REFERENCES [issues]([id])
);
CREATE INDEX [idx_issue_comments_issue]
    ON [issue_comments] ([issue]);
CREATE INDEX [idx_issue_comments_user]
    ON [issue_comments] ([user]);
Powered by Datasette · Queries took 11.496ms · About: xarray-datasette