home / github / issue_comments

Menu
  • GraphQL API
  • Search all tables

issue_comments: 786759897

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/2799#issuecomment-786759897 https://api.github.com/repos/pydata/xarray/issues/2799 786759897 MDEyOklzc3VlQ29tbWVudDc4Njc1OTg5Nw== 17484729 2021-02-26T16:43:23Z 2021-02-26T16:43:23Z NONE

Hi,

I'm working on a machine learning application where I want to stream data and use xarray containers to store them in a buffer (with an additional "lag" dimension) and guaranty good alignement of the coordinates on various dimensions of the streamed data. Doing so, I noticed that the version of my code working with xarray is very slow when compared to a pure numpy implementation (with no coordinate alignement) or even an implementation with deque+pandas. I think the performance issue that I noticed is basically the same observation than the ones of this issue.

I have the impression that for this kind of applications or more generally for intensive algorithmic usages, also as stated at the begining of this issue, a light (with less functionalities and checks) and fast version of xarray DataArray and Dataset containers could be developped.

Do you think this could be something doable in the scope of xarray? Would it be preferable to create a dedicated library?

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  416962458
Powered by Datasette · Queries took 3.727ms · About: xarray-datasette