id,node_id,number,title,user,state,locked,assignee,milestone,comments,created_at,updated_at,closed_at,author_association,active_lock_reason,draft,pull_request,body,reactions,performed_via_github_app,state_reason,repo,type 1379000408,I_kwDOAMm_X85SMeBY,7057,Hook for better control over copy behavior with duck-arrays?,12912489,open,0,,,0,2022-09-20T08:11:12Z,2022-09-20T08:12:22Z,,NONE,,,,"### Context By using `copy(deep=False)` custom operations may avoid copying large amounts of data but can modify, e.g., coord dicts: ```python tmp = da.copy(deep=False) del tmp.coords['abc'] # Use tmp ``` When `da` wraps a duck-array with substructure the current implementation is insufficient: ```python tmp = da.copy(deep=False) # Imagine a duck-array similar to numpy.ma but with a dict of masks del tmp.data.masks['abc'] # Bad: breaks `da` # Use tmp ``` ### Describe the solution you'd like Currently there does not appear a solution to this, unless we know details about the duck array. Therefore, I wonder if we need an additional ""hook"" that duck-arrays may provide, which could be called by Xarray to make a non-deep copy? ```python class MyDuckArray: def _copy_shallow_(self): # TODO: better name """"""Copy everything except buffers"""""" ``` ```python # in xarray.Variable def copy(self, deep=True): if deep: data = copy.deepcopy(data) elif hasattr(data, '_copy_shallow_'): data = data._copy_shallow_() ``` ### Additional context This is the current implementation for `Variable`. There is no operation for `deep=False`: https://github.com/pydata/xarray/blob/716973e41060184beebd64935afe196a805ef481/xarray/core/variable.py#L956-L957","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/7057/reactions"", ""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,,13221727,issue 520815068,MDU6SXNzdWU1MjA4MTUwNjg=,3509,"NEP 18, physical units, uncertainties, and the scipp library?",12912489,closed,0,,,3,2019-11-11T08:45:29Z,2022-09-09T13:10:47Z,2022-09-09T13:10:47Z,NONE,,,,"This is an idea and meant as a discussion starter on an potential route to providing support for physical units and propagation of uncertainties for `xarray`. ### Context 1. NEP 18 (which, as far as I understand, was pushed by you guys, for similar purposes) provides means to combine such features with `xarray` using `__array_function__`, if the underlying array implementation supports it. 2. I am working on [scipp](https://scipp.readthedocs.io/en/latest/), which (based on a decision I may or may not regret in the future) is reimplementing a lot of features that `xarray` provides, plus some additional features. Two of these features are physical units and propagation of uncertainties. - `scipp.Variable` is essentially equivalent to a `numpy` array with a unit, dimension labels, and an optional array of uncertainties. [*] - `scipp` implements basic arithmetic operations (and some more) for `scipp.Variable`, including efficient propagation of uncertainties. [*] *Caveat: `scipp`'s current unit implementation is static an would probably need to be replaced for becoming useful for a wider audience.* ### Idea and questions Can we implement the `__array_function__` protocol for `scipp.Variable` so it can be used with `xarray`? As far as I can tell this would simply be a lightweight wrapper. - Did I understand `__array_function__` correctly? - Is there anything else I should be aware of? - Would anyone be interested in this? This would amount to using the lower-level parts of `scipp` which is quite compact and can be extended to support more data types and more operations in a relatively simple manner (requiring recompilation, since it is written in C++).","{""url"": ""https://api.github.com/repos/pydata/xarray/issues/3509/reactions"", ""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,completed,13221727,issue