html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/3981#issuecomment-1243222416,https://api.github.com/repos/pydata/xarray/issues/3981,1243222416,IC_kwDOAMm_X85KGhGQ,12912489,2022-09-12T04:59:42Z,2022-09-12T04:59:42Z,NONE,"I note that `xarray.Variable` also provides `attrs`. Would it make sense to separate this aspect from the labelled dims? That is, instead of extracting this as a single library, turn it into two, such that users can pick one or both depending on their needs.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-1243218951,https://api.github.com/repos/pydata/xarray/issues/3981,1243218951,IC_kwDOAMm_X85KGgQH,12912489,2022-09-12T04:51:23Z,2022-09-12T04:55:13Z,NONE,"This is something I am getting more and more interested in. We ([scipp](https://scipp.github.io)) currently have a C++ implementation (with Pything bindings) of a simpler version of `xarray.Variable`. I am starting considerations of moving more of this to the Python side. So I would like to hear about the status of this? While I am still far from having reached a conclusion (or convincing anyone here to support this), investing in technology that is adopted and carried by the community is considered important here. In other words, we may in principle be able to help out and invest some time into this. One important precondition would be full compatibility with other custom array containers: For our applications we do not just need to add labelled axes, but also units, masks, bin edges, and ragged data support. I am currently toying with the idea of a ""stack"" of Python array libraries (I guess you would call them duck arrays?) that add these features one by one, selectively, but can all be used also independently --- unlike Scipp, where you get all or nothing, and lose the ability of using NumPy (or other) array libraries under the hood. Each of those libraries could be small and simple, focussing one just one specific aspect, but everything should be composable. For example, we can imagine a `Variable` with a `pint` array for having units as well as labelled dimensions.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-986178849,https://api.github.com/repos/pydata/xarray/issues/3981,986178849,IC_kwDOAMm_X846x-Uh,6531703,2021-12-05T07:09:22Z,2021-12-05T07:09:22Z,NONE,I can't make the calls next week - but maybe @andy-sweet can report back and we'll take it from there. Thanks!!,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-985824018,https://api.github.com/repos/pydata/xarray/issues/3981,985824018,IC_kwDOAMm_X846wnsS,2443309,2021-12-03T20:59:27Z,2021-12-03T20:59:27Z,MEMBER,@andy-sweet - please do join the next call. I've added it to the meeting agenda. ,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-985816997,https://api.github.com/repos/pydata/xarray/issues/3981,985816997,IC_kwDOAMm_X846wl-l,2608297,2021-12-03T20:46:37Z,2021-12-03T20:46:37Z,NONE,"> We have a bi-weekly developer call on Wednesday mornings (#4001), one idea would be devote 10-15 minutes of our next meeting to this topic. Is that something you and/or @andy-sweet would be up for joining? Sounds good to me. Looks like the next one is on December 7th at 13:30 UTC? If so, should I just jump on the Zoom meeting or do I need to add an item to the agenda? I'm still fairly new to xarray and have mostly poked around a little at the feasibility of using `DataSet` and `DataArray` in napari, but I think that just about gives me enough time to look at `Variable` in a bit more detail to understand how much that would help us.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-985051449,https://api.github.com/repos/pydata/xarray/issues/3981,985051449,IC_kwDOAMm_X846trE5,2443309,2021-12-02T22:25:29Z,2021-12-02T22:25:39Z,MEMBER,"Hi @sofroniewn - This is certainly something we still want to work on (see [this section of our current roadmap](http://xarray.pydata.org/en/stable/roadmap.html#labeled-array-without-coordinates) and a [more detailed proposal that included work in this area](https://doi.org/10.5281/zenodo.5484175)). I actually think the relevant part of the linked proposal is the best we have for a working plan here (text copied from doc below): > ### Xvariable: New lightweight Variable API (labeled arrays without coordinates) > This work area is about cleaning up Xarray’s internals, and allowing our low-level “Variable” data structure to be usable by other projects in the scientific Python ecosystem. We have identified the following key tasks and deliverables: > > a. Formalize Xarray’s contract for valid data inside “Variable”, and remove/replace some legacy features that would be hard to justify for a generic library: > 1. Move the “encoding” attribute from “Variable” onto a new “duck array” class that can be used inside a “Variable” (#5082) > 2. Expose Xarray’s internal model for “explicit array indexing” as a public API. Xarray uses this feature because supporting the full complexity of NumPy’s full indexing API is hard for many array implementations. > > b. Separate out and possibly rename/re-brand to Xvariable parts of Xarray internals into new projects. This will increase their visibility, find new users for these tools and improve the maintainability of Xarray itself. > 1. The ”Variable” class will move into a separate project that only depends upon NumPy, and that Xarray in-turn will depend upon. We hope this project will be of interest to users interested in simpler tools than Xarray (#3981). > 2. The new package will support indexing and a limited series of other operations lazily on arrays loaded from disk or remote storage, without loading the entire array into memory. This project is of interest to other projects such as Napari (#5081). > > c. API stabilization, code consolidation and maintenance of the external project. We have a bi-weekly developer call on Wednesday mornings (#4001), one idea would be devote 10-15 minutes of our next meeting to this topic. Is that something you and/or @andy-sweet would be up for joining? ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-984906753,https://api.github.com/repos/pydata/xarray/issues/3981,984906753,IC_kwDOAMm_X846tHwB,6531703,2021-12-02T18:54:57Z,2021-12-02T18:54:57Z,NONE,"hi all, just wanted to check in if there's been any further thoughts/ progress here. We're having more and more need for an `xarray-lite` style package - so just simple around dimension naming/indexing - and we potentially have some engineering time to devote to this effort cc @andy-sweet. Would it make sense for the interested parties to hop on a call sometime to discuss?","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-877607680,https://api.github.com/repos/pydata/xarray/issues/3981,877607680,MDEyOklzc3VlQ29tbWVudDg3NzYwNzY4MA==,67612,2021-07-10T09:45:04Z,2021-07-10T09:45:04Z,NONE,@shoyer - thanks for the feedback. I guess this means that it's unlikely this will be ready in time for our own CZI grant to finish (around June 2022)?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-877464691,https://api.github.com/repos/pydata/xarray/issues/3981,877464691,MDEyOklzc3VlQ29tbWVudDg3NzQ2NDY5MQ==,1217238,2021-07-09T21:25:25Z,2021-07-09T21:25:25Z,MEMBER,"We wrote a proposal to build an ""xarray-lite"" as part of a recent CZI application. We're still waiting to hear back, but if that's funded we'd definitely love to work with you on this. On Fri, Jul 9, 2021 at 12:16 PM Matthew Brett ***@***.***> wrote: > We were just talking about this over at https://github.com/nipy/nibabel - > because we are about commit ourselves to an array-axis-labelling API. Is > xarray-lite on the near or the distant horizon? Should we wait, to make > our decisions? Mentioning @effigies because > he reminded me about this thread. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-877403316,https://api.github.com/repos/pydata/xarray/issues/3981,877403316,MDEyOklzc3VlQ29tbWVudDg3NzQwMzMxNg==,67612,2021-07-09T19:15:52Z,2021-07-09T19:15:52Z,NONE,"We were just talking about this over at https://github.com/nipy/nibabel - because we are about commit ourselves to an array-axis-labelling API. Is `xarray-lite` on the near or the distant horizon? Should we wait, to make our decisions? Mentioning @effigies because he reminded me about this thread.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-808791582,https://api.github.com/repos/pydata/xarray/issues/3981,808791582,MDEyOklzc3VlQ29tbWVudDgwODc5MTU4Mg==,13301940,2021-03-27T19:36:37Z,2021-03-27T19:36:37Z,MEMBER,"> Perhaps we could all this xarr to denote a minimal version of xarray, unless that's too punny? I've heard folks referring to this proposal as `xarray-lite`, and I think it'd be a good name, too. ","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-808677644,https://api.github.com/repos/pydata/xarray/issues/3981,808677644,MDEyOklzc3VlQ29tbWVudDgwODY3NzY0NA==,1217238,2021-03-27T07:18:23Z,2021-03-27T07:18:23Z,MEMBER,"Thinking about this a little more, I would lean towards the separate package that xarray could depend upon. Perhaps we could all this `xarr` to denote a minimal version of `xarray`, unless that's too punny?","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-665812548,https://api.github.com/repos/pydata/xarray/issues/3981,665812548,MDEyOklzc3VlQ29tbWVudDY2NTgxMjU0OA==,1217238,2020-07-29T17:58:50Z,2020-07-29T17:58:50Z,MEMBER,"> BUT @shoyer has been consistent against making Variable public, so maybe the best option is to fork it out to `xarray-contrib/variable`? `Variable` is a public API, which just don't encourage it for (most) users. I think it would be relatively straightforward to expose `xarray.Variable` without the pandas dependency. We currently use pandas a little bit for input normalization in `Variable` and methods like `isnull()`, but we could likely add slower fallback option (or just not expose these features unless pandas is installed).","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-665746983,https://api.github.com/repos/pydata/xarray/issues/3981,665746983,MDEyOklzc3VlQ29tbWVudDY2NTc0Njk4Mw==,2448579,2020-07-29T15:52:17Z,2020-07-29T15:57:00Z,MEMBER,"Another possible outcome is that pandas becomes optional as part of the indexing refactor, and we expose Variable as public API. BUT @shoyer has been consistent against making Variable public, so maybe the best option is to fork it out to `xarray-contrib/variable`? EDIT: The git log for `variable.py` shows really minor changes over the past year or two (https://github.com/pydata/xarray/commits/master/xarray/core/variable.py) — mostly adding new method (e.g. pad & quantile) and some NEP-18 work. So it seems pretty stable and perhaps not much effort to exchange future patches between xarray and the fork. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-665730527,https://api.github.com/repos/pydata/xarray/issues/3981,665730527,MDEyOklzc3VlQ29tbWVudDY2NTczMDUyNw==,5635139,2020-07-29T15:23:09Z,2020-07-29T15:23:09Z,MEMBER,"What would the options be for moving this forward? - Try / Except import blocks, like we do for other libraries - A separate package (could be in the same repo though) - ?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-665533405,https://api.github.com/repos/pydata/xarray/issues/3981,665533405,MDEyOklzc3VlQ29tbWVudDY2NTUzMzQwNQ==,67612,2020-07-29T08:56:23Z,2020-07-29T08:56:23Z,NONE,"Just to add - we at [Nibabel](https://github.com/nipy/nibabel) are very interested in adding labelled arrays with Xarray, but for us, the Pandas dependency is a serious problem. We're a base library for reading brain imaging formats, and we sit at the bottom of several imaging stacks, so it is very important to us that we don't introduce heavy dependencies - because we pass these on to all the libraries that depend on us. We've looked enviously at Xarray for a while, but the Pandas dependency is a serious-enough problem that we've held off from using it. Just for example, the Pandas dependency of Xarray was the reason that I [was working](https://github.com/BIDS/datarray/graphs/contributors) on [Datarray](https://pypi.org/project/datarray), to see if we could use that instead (we couldn't).","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-665216474,https://api.github.com/repos/pydata/xarray/issues/3981,665216474,MDEyOklzc3VlQ29tbWVudDY2NTIxNjQ3NA==,83442,2020-07-28T18:54:11Z,2020-07-28T18:54:11Z,NONE,"To add another use case, the [NiBabel](https://github.com/nipy/nibabel/) package has considered how to label axes (https://github.com/nipy/nibabel/wiki/BIAP6, https://github.com/nipy/nibabel/issues/412), but it's fallen by the wayside. We considered xarray when it was still xray, but the pandas dependency has always been a sticking point. This is partially due to a desire to keep dependencies minimal, and partly due to the size of pandas causing significant overhead at import time for what is a relatively small component. If it's useful to go further into our use case, we can, but this is just to put in a vote for making pandas an optional dependency, if possible.","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-615494816,https://api.github.com/repos/pydata/xarray/issues/3981,615494816,MDEyOklzc3VlQ29tbWVudDYxNTQ5NDgxNg==,2443309,2020-04-17T22:40:59Z,2020-04-17T22:40:59Z,MEMBER,"Thanks @amueller! You're not the only group that has voiced interest in an index-free labeled array so I think its still worth discussing. > I think right now we're most concerned about sparse data representations (and I was considering asking you folks if you'd support scipy.sparse ; We have recently added support for [pydata/sparse](https://sparse.pydata.org/en/latest/). We need to document this better (#3484). This work is not 100% complete (https://github.com/pydata/xarray/issues/3213) and is part of Xarray's larger goal of [supporting duck arrays](https://github.com/pydata/xarray/projects/2). Please chime in on one of those issues so we can go a bit deeper.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880 https://github.com/pydata/xarray/issues/3981#issuecomment-615490533,https://api.github.com/repos/pydata/xarray/issues/3981,615490533,MDEyOklzc3VlQ29tbWVudDYxNTQ5MDUzMw==,449558,2020-04-17T22:24:36Z,2020-04-17T22:24:36Z,NONE,"FYI the conversation on sklearn is far from resolved, and at this point I think the added pandas dependency is not what will keep us from using xarray. I think right now we're most concerned about sparse data representations (and I was considering asking you folks if you'd support scipy.sparse ;)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,602256880