home / github

Menu
  • Search all tables
  • GraphQL API

issues

Table actions
  • GraphQL API for issues

4 rows where user = 8419157 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: comments, created_at (date), updated_at (date), closed_at (date)

state 2

  • closed 3
  • open 1

type 1

  • issue 4

repo 1

  • xarray 4
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
659142025 MDU6SXNzdWU2NTkxNDIwMjU= 4235 Dataset plot line DancingQuanta 8419157 open 0     5 2020-07-17T10:51:24Z 2022-04-18T06:59:15Z   NONE      

Is your feature request related to a problem? Please describe.

Dataset have one plotting method Dataset.plot.scatter. This uses matplotlb's ax.scatter which have a limited presentation configuration (eg linestyle and marker). I have 2D X and Y data which I like to plot against each other. The DataArray.plot.line works well if I only use a 1D X data.

Describe the solution you'd like

A new plotting method Dataset.plot.line based on matplotlib's ax.plot which will allow me to plot 2D X and Y against each other with flexible presentation configuration. In other to be able to produce 1D lines, this must demand a required 3D dimension as an argument (eg a choice of hue, col or others).

I suspect that as increase in number of dimensions from 1D to 2D for both variables and later to ND, there will be many ways to interpret the arguments given to the plotting function,. However, the fact the method name is line in indicate that it must produce 1D lines, and use other dimensions to create other features such as legend or subplots.

Having seen the code for the Dataset.plot.scatter I am keen to submit a PR to add Dataset.plot.line.

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4235/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    xarray 13221727 issue
553672182 MDU6SXNzdWU1NTM2NzIxODI= 3715 ValueError: buffer source array is read-only with apply_ufunc DancingQuanta 8419157 closed 0     5 2020-01-22T17:00:23Z 2022-04-18T02:18:47Z 2022-04-18T02:18:47Z NONE      

I am trying to use scipy.signal.find_peaks function within apply_ufunc but fails with ValueError: buffer source array is read-only.

MCVE Code Sample

```python import numpy as np import xarray as xr from scipy.signal import find_peaks

Generate waveform

x = (np.sin(2np.pi(2*np.linspace(2,10,1000))np.arange(1000)/48000) + np.random.normal(0, 1, 1000) * 0.15)

Find peaks non-xarray way

peaks, _ = find_peaks(x, prominence=1) print(peaks)

Cast waveform to xr.DataArray

x = xr.DataArray(x, dims='time')

Duplicate data along a new dimension

rep = xr.DataArray(range(11), dims='repeat') x = (x.broadcast_like(rep).assign_coords(repeat=rep))

def process_peaks(arr): # Apply find_peaks peaks, _ = find_peaks(arr, prominence=1) return peaks

Apply function to array

results = xr.apply_ufunc( process_peaks, x, input_core_dims=[['time']], output_core_dims=[['peaks']], vectorize=True )

Should show repeats of peak results

print(results) ```

Expected Output

In the MCvE above, there are two print statements, The first print out the results of the peak finding without involving xarray. The second print statement prints out the xr.DataArray of same peak results repeated along a dimension repeat.

Problem Description

The function scipy.signal.find_peaks threw an error ValueError: buffer source array is read-only. See the full traceback below ```


ValueError Traceback (most recent call last) <ipython-input-43-1a80f67560a6> in <module> 35 input_core_dims=[['time']], 36 output_core_dims=[['peaks']], ---> 37 vectorize=True 38 ) 39

c:\users\at17\repos\xarray\xarray\core\computation.py in apply_ufunc(func, input_core_dims, output_core_dims, exclude_dims, vectorize, join, dataset_join, dataset_fill_value, keep_attrs, kwargs, dask, output_dtypes, output_sizes, *args) 1040 join=join, 1041 exclude_dims=exclude_dims, -> 1042 keep_attrs=keep_attrs, 1043 ) 1044 elif any(isinstance(a, Variable) for a in args):

c:\users\at17\repos\xarray\xarray\core\computation.py in apply_dataarray_vfunc(func, signature, join, exclude_dims, keep_attrs, args) 230 231 data_vars = [getattr(a, "variable", a) for a in args] --> 232 result_var = func(data_vars) 233 234 if signature.num_outputs > 1:

c:\users\at17\repos\xarray\xarray\core\computation.py in apply_variable_ufunc(func, signature, exclude_dims, dask, output_dtypes, output_sizes, keep_attrs, args) 599 "apply_ufunc: {}".format(dask) 600 ) --> 601 result_data = func(input_data) 602 603 if signature.num_outputs == 1:

~.conda\envs\asi\lib\site-packages\numpy\lib\function_base.py in call(self, args, *kwargs) 2089 vargs.extend([kwargs[_n] for _n in names]) 2090 -> 2091 return self._vectorize_call(func=func, args=vargs) 2092 2093 def _get_ufunc_and_otypes(self, func, args):

~.conda\envs\asi\lib\site-packages\numpy\lib\function_base.py in _vectorize_call(self, func, args) 2155 """Vectorized call to func over positional args.""" 2156 if self.signature is not None: -> 2157 res = self._vectorize_call_with_signature(func, args) 2158 elif not args: 2159 res = func()

~.conda\envs\asi\lib\site-packages\numpy\lib\function_base.py in _vectorize_call_with_signature(self, func, args) 2196 2197 for index in np.ndindex(broadcast_shape): -> 2198 results = func((arg[index] for arg in args)) 2199 2200 n_results = len(results) if isinstance(results, tuple) else 1

<ipython-input-43-1a80f67560a6> in process_peaks(arr) 27 28 # Finally execute find_peaks ---> 29 peaks, _ = find_peaks(arr, prominence=1) 30 return peaks 31

~.conda\envs\asi\lib\site-packages\scipy\signal_peak_finding.py in find_peaks(x, height, threshold, distance, prominence, width, wlen, rel_height, plateau_size) 937 raise ValueError('distance must be greater or equal to 1') 938 --> 939 peaks, left_edges, right_edges = _local_maxima_1d(x) 940 properties = {} 941

_peak_finding_utils.pyx in scipy.signal._peak_finding_utils._local_maxima_1d()

~.conda\envs\asi\lib\site-packages\scipy\signal_peak_finding_utils.cp37-win_amd64.pyd in View.MemoryView.memoryview_cwrapper()

~.conda\envs\asi\lib\site-packages\scipy\signal_peak_finding_utils.cp37-win_amd64.pyd in View.MemoryView.memoryview.cinit()

ValueError: buffer source array is read-only ```

I have a conversation with some people on Gitter about it and a workaround was proposed, adding this line of code before find_peaks arr = arr.copy() I am not sure if a read-only array is a view or not. This appears to be a known problem to some people but there are no documentation on this limitation of apply_ufunc. It may be unlikely that a copy operation will be added to apply_ufunc to deal with this issue internally. So the way forwards is to add documentation to apply_ufunc about this issue and the workaround.

Output of xr.show_versions()

``` INSTALLED VERSIONS


commit: None python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 78 Stepping 3, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3

xarray: 0.14.1+50.ge0fd4805 pandas: 0.25.3 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.7.4 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nc_time_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.1.2 cartopy: None seaborn: None numbagg: None setuptools: 45.0.0.post20200113 pip: 19.3.1 conda: None pytest: None IPython: 7.11.1 sphinx: None ​```

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3715/reactions",
    "total_count": 2,
    "+1": 2,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed xarray 13221727 issue
603309899 MDU6SXNzdWU2MDMzMDk4OTk= 3985 xarray=1.15.1 regression: Groupby drop multi-index DancingQuanta 8419157 closed 0     4 2020-04-20T15:05:51Z 2021-02-16T15:59:46Z 2021-02-16T15:59:46Z NONE      

I have written a function process_stacked_groupby that stack all but one dimension of a dataset/dataarray and perform groupby-apply-combine on the stacked dimension. However, after upgrading to 0.15.1, the function cease to work.

MCVE Code Sample

```python import xarray as xr

Dimensions

N = xr.DataArray(np.arange(100), dims='N', name='N') reps = xr.DataArray(np.arange(5), dims='reps', name='reps') horizon = xr.DataArray([1, -1], dims='horizon', name='horizon') horizon.attrs = {'long_name': 'Horizonal', 'units': 'H'} vertical = xr.DataArray(np.arange(1, 4), dims='vertical', name='vertical') vertical.attrs = {'long_name': 'Vertical', 'units': 'V'}

Variables

x = xr.DataArray(np.random.randn(len(N), len(reps), len(horizon), len(vertical)), dims=['N', 'reps', 'horizon', 'vertical'], name='x') y = x * 0.1 y.name = 'y'

Merge x, y

data = xr.merge([x, y])

Assign coords

data = data.assign_coords(reps=reps, vertical=vertical, horizon=horizon)

Function that stack all but one diensions and groupby over the stacked dimension.

def process_stacked_groupby(ds, dim, func, *args):

# Function to apply to stacked groupby
def apply_fn(ds, dim, func, *args):

    # Get groupby dim
    groupby_dim = list(ds.dims)
    groupby_dim.remove(dim)
    groupby_var = ds[groupby_dim]

    # Unstack groupby dim
    ds2 = ds.unstack(groupby_dim).squeeze()

    # perform function
    ds3 = func(ds2, *args)

    # Add mulit-index groupby_var to result
    ds3 = (ds3
           .reset_coords(drop=True)
           .assign_coords(groupby_var)
           .expand_dims(groupby_dim)
         )
    return ds3

# Get list of dimensions
groupby_dims = list(ds.dims)

# Remove dimension not grouped
groupby_dims.remove(dim)

# Stack all but one dimensions
stack_dim = '_'.join(groupby_dims)
ds2 = ds.stack({stack_dim: groupby_dims})

# Groupby and apply
ds2 = ds2.groupby(stack_dim, squeeze=False).map(apply_fn, args=(dim, func, *args))

# Unstack
ds2 = ds2.unstack(stack_dim)

# Restore attrs
for dim in groupby_dims:
    ds2[dim].attrs = ds[dim].attrs

return ds2

Function to apply on groupby

def fn(ds): return ds

Run groupby with applied function

data.pipe(process_stacked_groupby, 'N', fn) ```

Expected Output

Prior to xarray=0.15.0, the above code produce a result that I wanted.

The function should be able to 1. stack chosen dimensions 2. groupby the stacked dimension 3. apply a function on each group a. The function actually passes along another function with unstacked group coord b. Add multi-index stacked group coord back to the results of this function 4. combine the groups 5. Unstack stacked dimension

Problem Description

After upgrading to 0.15.1, the above code stopped working. The error occurred at the line # Unstack ds2 = ds2.unstack(stack_dim) with ValueError: cannot unstack dimensions that do not have a MultiIndex: ['horizon_reps_vertical']. This is on 5th step where the resulting combined object was found not to contain any multi-index. Somewhere in the 4th step, the combination of groups have lost the multi-index stacked dimension.

Versions

0.15.1

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3985/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed xarray 13221727 issue
553766743 MDU6SXNzdWU1NTM3NjY3NDM= 3716 Feature request: apply_ufunc create dataset from multi-return function DancingQuanta 8419157 closed 0     1 2020-01-22T20:10:43Z 2020-01-25T00:00:12Z 2020-01-25T00:00:12Z NONE      

I wanted to pass a list of variable names to apply_ufunc which then be used to convert a returned tuple of DataArrays into a dataset with names from the list.

I could create a PR but I need to ask due to the large possible arguments to and returns from apply_ufunc which may not match the use case with returned tuple of DataArrays.

{
    "url": "https://api.github.com/repos/pydata/xarray/issues/3716/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  completed xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options:

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);
Powered by Datasette · Queries took 23.597ms · About: xarray-datasette