issues

Table actions

2 rows where repo = 13221727, state = "closed" and user = 1478822 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

type 1

state 1

  • closed · 2 ✖

repo 1

  • xarray · 2 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association active_lock_reason draft pull_request body reactions performed_via_github_app state_reason repo type
685613931 MDU6SXNzdWU2ODU2MTM5MzE= 4374 Xarray improper handling of Fillvalues when converting to Numpy array Marston 1478822 closed 0     2 2020-08-25T16:11:53Z 2020-08-25T16:39:34Z 2020-08-25T16:39:34Z NONE      

I use xarray to access some geospatial data from an opendap server using: xrObject = xr.open_dataset(opendapUrl) The problem is that when I need to pass this data to Numpy, since GDAL only accepts Numpy arrays, xarray passes NaNs and not the proper fill value which is listed in the attributes. This to me is a bug. Apologies for not providing a code snippet but here is the print out:

I'm running this using python 3.8.3 on a Centos 6 machine using xarray 0.16.0 and netcdf4 version 1.5.3

** Results *** ``` Opening file: https://opendapserver/2001/001/3B-HHR-E.HDF5 Access of opendap metadata successful Max: <xarray.DataArray 'precipitationCal' ()> array(102.84, dtype=float32) Min: <xarray.DataArray 'precipitationCal' ()> array(0., dtype=float32) <xarray.DataArray 'precipitationCal' (time: 1, lat: 1800, lon: 3600)> array([[[nan, nan, nan, ..., nan, nan, nan], [nan, nan, nan, ..., nan, nan, nan], [nan, nan, nan, ..., nan, nan, nan], ..., [nan, nan, nan, ..., nan, nan, nan], [nan, nan, nan, ..., nan, nan, nan], [nan, nan, nan, ..., nan, nan, nan]]], dtype=float32) Coordinates: * lat (lat) float32 -89.95 -89.85 -89.75 -89.65 ... 89.75 89.85 89.95 * lon (lon) float32 -179.95 -179.85 -179.75 ... 179.75 179.85 179.95 * time (time) object 2001-01-01 00:00:00 Attributes: DimensionNames: time,lat,lon Units: mm/hr units: mm/hr CodeMissingValue: -9999.9 origname: precipitationCal fullnamepath: /Grid/precipitationCal OldDimensionNames: Dataset was transposed.

Print Numpy array after command: dataSlice = np.array(xrArray.squeeze()); print(dataSlice)

[[nan nan nan ... nan nan nan] [nan nan nan ... nan nan nan] [nan nan nan ... nan nan nan] ... [nan nan nan ... nan nan nan] [nan nan nan ... nan nan nan] [nan nan nan ... nan nan nan]] ```

So the problem is that Xarray is converting to a Numpy array and not filling in the fill values from the CodeMissingValue attribute. It is also not very easy to print the xarray raw data in a similar manner Numpy masked array does using the filled() method. I think the developers can agree that it is a good thing for xarray to pass the data to another structure as it received it, i.e., without replacing the fill values.

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/4374/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed xarray 13221727 issue
217485308 MDU6SXNzdWUyMTc0ODUzMDg= 1334 time dimension Marston 1478822 closed 0     9 2017-03-28T08:33:55Z 2019-03-28T15:22:06Z 2019-03-28T15:22:06Z NONE      

Hi,

I'm new to xarray and would like to use it to process some data. For this task I need to access the time dimension and get the date in terms of year, month, and hour. I can do this if the time variable is a netcdf object but xarray does not return this object type.

For example:

import netcdf4 as nc

idata = nc.Dataset(ifile,'r') time = idata.variables['time'] dtime = num2date(time[:],time.units) hour = time[0].hour

How could access the time sub-variables in a similar manner?

 
{
    "url": "https://api.github.com/repos/pydata/xarray/issues/1334/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed xarray 13221727 issue

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [active_lock_reason] TEXT,
   [draft] INTEGER,
   [pull_request] TEXT,
   [body] TEXT,
   [reactions] TEXT,
   [performed_via_github_app] TEXT,
   [state_reason] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
);
CREATE INDEX [idx_issues_repo]
    ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
    ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
    ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
    ON [issues] ([user]);