{"database": "github", "table": "issue_comments", "is_view": false, "human_description_en": "where issue = 479942077, \"updated_at\" is on date 2022-01-16 and user = 40465719 sorted by updated_at descending", "rows": [["https://github.com/pydata/xarray/issues/3213#issuecomment-1013887301", "https://api.github.com/repos/pydata/xarray/issues/3213", 1013887301, "IC_kwDOAMm_X848brFF", 40465719, "2022-01-16T14:35:29Z", "2022-01-16T14:40:13Z", "NONE", "I would prefer to retain the dense representation, but with tricks to keep the data of sparse type in memory.\r\n\r\nLook at the following example with pandas multiindex & sparse dtype:\r\n![image](https://user-images.githubusercontent.com/40465719/149663769-4b9f625d-0a7d-4df7-9157-83795abbf3f3.png)\r\n\r\nThe dense data uses ~40 MB of memory, while the dense representation with sparse dtypes uses only ~0.5 kB of memory!\r\n\r\nAnd while you can import dataframes with the sparse=True keyword, the size seems to be displayed inaccurately (both are the same size?), and we cannot examine the data like we can with pandas multiindex + sparse dtype:\r\n![image](https://user-images.githubusercontent.com/40465719/149663790-927e190a-adff-45ea-a7c7-430cb2d3b8a7.png)\r\n\r\nBesides, a lot of operations are not available on sparse xarray data variables (i.e. if I wanted to group by price level for ffill & downsampling):\r\n![image](https://user-images.githubusercontent.com/40465719/149664636-4fa9696f-de49-4886-a73c-6f20b622ae5c.png)\r\n\r\nSo, it would be nice if xarray adopted pandas\u2019 approach of unstacking sparse data.\r\n\r\nIn the end, you could extract all the non-NaN values and write them to a sparse storage format, such as TileDB sparse arrays.\r\ncc: @stavrospapadopoulos", "{\"total_count\": 2, \"+1\": 2, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", null, 479942077]], "truncated": false, "filtered_table_rows_count": 1, "expanded_columns": [], "expandable_columns": [[{"column": "issue", "other_table": "issues", "other_column": "id"}, "title"], [{"column": "user", "other_table": "users", "other_column": "id"}, "login"]], "columns": ["html_url", "issue_url", "id", "node_id", "user", "created_at", "updated_at", "author_association", "body", "reactions", "performed_via_github_app", "issue"], "primary_keys": ["id"], "units": {}, "query": {"sql": "select html_url, issue_url, id, node_id, user, created_at, updated_at, author_association, body, reactions, performed_via_github_app, issue from issue_comments where \"issue\" = :p0 and date(\"updated_at\") = :p1 and \"user\" = :p2 order by updated_at desc limit 101", "params": {"p0": "479942077", "p1": "2022-01-16", "p2": "40465719"}}, "facet_results": {"author_association": {"name": "author_association", "type": "column", "hideable": false, "toggle_url": "/github/issue_comments.json?issue=479942077&updated_at__date=2022-01-16&user=40465719", "results": [{"value": "NONE", "label": "NONE", "count": 1, "toggle_url": "http://xarray-datasette.fly.dev/github/issue_comments.json?issue=479942077&updated_at__date=2022-01-16&user=40465719&author_association=NONE", "selected": false}], "truncated": false}, "user": {"name": "user", "type": "column", "hideable": false, "toggle_url": "/github/issue_comments.json?issue=479942077&updated_at__date=2022-01-16&user=40465719", "results": [{"value": 40465719, "label": "Material-Scientist", "count": 1, "toggle_url": "http://xarray-datasette.fly.dev/github/issue_comments.json?issue=479942077&updated_at__date=2022-01-16", "selected": true}], "truncated": false}, "issue": {"name": "issue", "type": "column", "hideable": false, "toggle_url": "/github/issue_comments.json?issue=479942077&updated_at__date=2022-01-16&user=40465719", "results": [{"value": 479942077, "label": "How should xarray use/support sparse arrays?", "count": 1, "toggle_url": "http://xarray-datasette.fly.dev/github/issue_comments.json?updated_at__date=2022-01-16&user=40465719", "selected": true}], "truncated": false}}, "suggested_facets": [{"name": "created_at", "type": "date", "toggle_url": "http://xarray-datasette.fly.dev/github/issue_comments.json?issue=479942077&updated_at__date=2022-01-16&user=40465719&_facet_date=created_at"}, {"name": "updated_at", "type": "date", "toggle_url": "http://xarray-datasette.fly.dev/github/issue_comments.json?issue=479942077&updated_at__date=2022-01-16&user=40465719&_facet_date=updated_at"}], "next": null, "next_url": null, "private": false, "allow_execute_sql": true, "query_ms": 24.39874690026045}