html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/3213#issuecomment-1014462537,https://api.github.com/repos/pydata/xarray/issues/3213,1014462537,IC_kwDOAMm_X848d3hJ,40465719,2022-01-17T12:20:18Z,2022-01-17T12:20:18Z,NONE,"I know. But having sparse data I can treat as if it were dense allows me to unstack without running out of memory, and then ffill & downsample the data in chunks:

![image](https://user-images.githubusercontent.com/40465719/149767712-43053b8d-ec87-4f3a-804d-14704ab785cf.png)

It would be nice if xarray automatically converted the data from sparse back to dense for doing operations on the chunks just like pandas does.

The picture shows that I'm already using nbytes to determine the size.
","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,479942077
https://github.com/pydata/xarray/issues/3213#issuecomment-1013887301,https://api.github.com/repos/pydata/xarray/issues/3213,1013887301,IC_kwDOAMm_X848brFF,40465719,2022-01-16T14:35:29Z,2022-01-16T14:40:13Z,NONE,"I would prefer to retain the dense representation, but with tricks to keep the data of sparse type in memory.

Look at the following example with pandas multiindex & sparse dtype:
![image](https://user-images.githubusercontent.com/40465719/149663769-4b9f625d-0a7d-4df7-9157-83795abbf3f3.png)

The dense data uses ~40 MB of memory, while the dense representation with sparse dtypes uses only ~0.5 kB of memory!

And while you can import dataframes with the sparse=True keyword, the size seems to be displayed inaccurately (both are the same size?), and we cannot examine the data like we can with pandas multiindex + sparse dtype:
![image](https://user-images.githubusercontent.com/40465719/149663790-927e190a-adff-45ea-a7c7-430cb2d3b8a7.png)

Besides, a lot of operations are not available on sparse xarray data variables (i.e. if I wanted to group by price level for ffill & downsampling):
![image](https://user-images.githubusercontent.com/40465719/149664636-4fa9696f-de49-4886-a73c-6f20b622ae5c.png)

So, it would be nice if xarray adopted pandas’ approach of unstacking sparse data.

In the end, you could extract all the non-NaN values and write them to a sparse storage format, such as TileDB sparse arrays.
cc: @stavrospapadopoulos","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,479942077