home / github / issue_comments

Menu
  • Search all tables
  • GraphQL API

issue_comments: 1169014257

This data as json

html_url issue_url id node_id user created_at updated_at author_association body reactions performed_via_github_app issue
https://github.com/pydata/xarray/issues/6733#issuecomment-1169014257 https://api.github.com/repos/pydata/xarray/issues/6733 1169014257 IC_kwDOAMm_X85Frb3x 2448579 2022-06-28T17:23:12Z 2022-06-28T17:26:17Z MEMBER

Yeah I think the issue is that the "CFMaskCoder" tries to repalce NaNs regardless of the dtype of the variable. Doing this creates a copy in this step: where(notnull(data), data, other).

https://github.com/pydata/xarray/blob/787a96c15161c9025182291b672b3d3c5548a6c7/xarray/coding/variables.py#L149

You should set FillValue to None after manually encoding to ints to skip the extra copy.

We should probably raise an error or at least a warning for integer dtypes and not-None FillValue


As for your initial question, we create a copy of the float array when replacing NaNs (does not happen in-place), then convert to int. So you'll need to account for 2x float array + 1x int array memory use.

{
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
  1286995366
Powered by Datasette · Queries took 0.688ms · About: xarray-datasette