issue_comments

Indeed. Thanks to all people involved!

Closed by #1750?

I think either way works. Probably easiest to fork this branch and then merge in the latest from master. On Thu, Nov 30, 2017 at 3:38 PM Neil Parley notifications@github.com wrote:

I think also I could make you a contributor or transfer the branch to someone. So that might be an option too.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/814#issuecomment-348225565, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1oT1qWWy-o5CJe6Y45Y-DsGhjTucks5s7swSgaJpZM4H9jGH .

I think also I could make you a contributor or transfer the branch to someone. So that might be an option too.

I'm happy to pick this up - I'm not sure how best to pick up someone else's pull request though. Am I best off forking nparley:master then resubmitting, or creating a new one and merging in the changes from nparley:master?

No worries! The great thing about GitHub is that it's pretty easy for someone else to pick up your work :)

On Tue, Jun 20, 2017 at 12:25 AM, Neil Parley notifications@github.com wrote:

Sorry guys I don't have time to do anything on this (or any open source stuff) at the moment.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/814#issuecomment-309667894, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1jC8Vrt_NKYbc4Z3-sgDpRAfaDqsks5sF3PXgaJpZM4H9jGH .

Sorry guys I don't have time to do anything on this (or any open source stuff) at the moment.

Yes, we would still love to get this in, though it might need a few other changes to work now. Could you open a new PR with the tests added back in? (Assuming Neil is not interested in picking this back up)

On Mon, Jun 19, 2017 at 8:58 AM, DWesl notifications@github.com wrote:

If you're still looking for the old tests, it looks like they disappeared in the last merge commit, f48de5 https://github.com/pydata/xarray/pull/814/commits/f48de5a3ea916b80a7bfc070c3c1c3549e931189#diff-48ab4ba033ad06981f566a2e2f561f5aL1819 .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/814#issuecomment-309484883, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1gi9-YLVTLayYcMiFTqVSUy1v729ks5sFpqkgaJpZM4H9jGH .

If you're still looking for the old tests, it looks like they disappeared in the last merge commit, f48de5.

This did used to have some tests, but they seem to have got lost somewhere along the way. I'll add the old tests back in, which would be a start.

This is looking pretty comprehensive to me. Could maybe do with a couple of tests perhaps, but otherwise :+1: .

Indeed, merging this is really just blocked on tests right now. We need at least something so we can verify that this basically works.

This is looking pretty comprehensive to me. Could maybe do with a couple of tests perhaps, but otherwise 👍 .

This adds the extra code that @pelson suggested but it would be nice to create a little unit test with a mixed Dimension and Auxiliary coordinates. Is there an easy way to create a simple cube in Iris that could test this?

@pelson @rhattersley any tips on handling the edge cases of cubes with some, but not all missing dim_coords?

I might consider something like (untested):

for dim in xrange(cube.ndim): try: dim_coord = cube.coord(dim_coords=True, dimensions=(dim,)) except iris.exceptions.CoordinateNotFoundError: index_coord = range(cube.shape[dim])

No worries, no rush here from my end :). On Sun, Jun 26, 2016 at 1:35 PM Neil Parley notifications@github.com wrote:

@shoyer https://github.com/shoyer I have not forgotten about this PR, I just been working a lot on a pandas PR updating the ci and it's been taking up a lot of my spare time.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/pull/814#issuecomment-228596914, or mute the thread https://github.com/notifications/unsubscribe/ABKS1tPi3K7IyPa2MwwQWDfnrwMemmetks5qPmQJgaJpZM4H9jGH .

@shoyer I have not forgotten about this PR, I just been working a lot on a pandas PR updating the ci and it's been taking up a lot of my spare time.

@pelson @rhattersley any tips on handling the edge cases of cubes with some, but not all missing dim_coords?

@nparley I would hate for this to get lost. I'm OK merging this before it covers every edge case -- let's just document the limitations and leave that for later cleanup.

OK, I was not paying attention here with #860 -- I assumed that the build problem was with the build including Iris, but it was actually the other build that started spontaneously failing.

I'm afraid I already merged that PR, so could you please try rebasing again? Sorry for the hassle! On the plus side, I think using conda-forge will ultimately work better than using all the separate channels.

@nparley Let's see if https://github.com/pydata/xarray/pull/860 works. I think that will probably solve the CI issue (at least on Travis-CI).

I have merged upstream and it seems to be failing but I don't think it's related to this PR, as a clean clone of master branch is also failing https://travis-ci.org/nparley/xarray/jobs/132872124

@rhattersley I think this function is cleaner without the cf_var_name argument, so I would probably remove it. That said, I usually try to avoid issuing warnings at all.

There's been a question over whether parse_cell_methods should really have the cf_var_name argument. It's only used if it raises a warning, and even then it's only used to help make the message more user friendly. In itself that doesn't seem like a big deal, but it did make me wonder if you'd rather if it didn't raise the warning at all?

@jhamman and I have a few minor suggestions, but otherwise this looks good!

This PR needs "what's new" and "api" entries in the docs.

@shoyer @pelson @rhattersley Is there anything you think could be added to this PR or is this ok for now?

OK this update uses iris.fileformats._pyke_rules.compiled_krb.fc_rules_cf_fc as @pelson mentioned in the comment above. I also added a cell method to the test

OK the latest commit tries to do the cell methods:

``` python

print(cube) sea_water_potential_temperature / (degC) (-- : 148; -- : 180) Auxiliary coordinates: latitude x x longitude x x Scalar coordinates: depth: 4.99994 m, bound=(0.0, 10.0) m time: 1-01-01 12:00:00 Attributes: Conventions: CF-1.5 Cell methods: mean: time_counter print(xarray.DataArray.from_iris(cube)) <xarray.DataArray u'votemper' (dim0: 148, dim1: 180)> array([[ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], ..., [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan], [ nan, nan, nan, ..., nan, nan, nan]], dtype=float32) Coordinates: deptht float64 5.0 time_counter object 1-01-01 12:00:00 nav_lat (dim0, dim1) float32 -78.1906 -78.1906 -78.1906 -78.1906 ... nav_lon (dim0, dim1) float32 80.0 82.0 83.9999 85.9999 87.9999 ... * dim0 (dim0) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 ... * dim1 (dim1) int64 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 ... Attributes: units: degC long_name: Temperature standard_name: sea_water_potential_temperature cell_methods: time_counter: mean Conventions: CF-1.5 print(xarray.DataArray.from_iris(cube).to_iris()) sea_water_potential_temperature / (degC) (dim0: 148; dim1: 180) Dimension coordinates: dim0 x - dim1 - x Auxiliary coordinates: latitude x x longitude x x Scalar coordinates: depth: 4.99993801117 m time: 1-01-01 12:00:00 Attributes: Conventions: CF-1.5 Cell methods: mean: time_counter ```

One other thing that I noticed from the latest example -- it would be nice to covert cell methods from Cubes into string attributes on DataArrays.

Is there any way to know for sure whether a coordinate has been automatically created? Or is the name the only clue?

I'm afraid the name is the only clue. My inclination would be not to do anything special here to "un-name" default coordinates, assuming Iris has a straightforward way to get rid of them. Dimensions like dim0 come up quickly in demo code but rarely in practice -- it's mostly a signal to the user that the should pick a meaningful name.

If this is the way to do things then does this comment need deleting?

Yes, that comment is out of date. I'm currently refactoring that code though, so let me take care care of it.

There's a separate argument about whether or not xarray should automatically convert cubes when passed into the xarray.DataArray constructor. We do this for pandas objects, but not any other labelled array types, currently.

The comment was just to mention that I had based my code on the code from cdms2 from #236 as suggested by @shoyer. I guess this comment would not be needed in the final merge.

Uses the same template as the cdms2 conversion.

If this is the way to do things then does this comment need deleting?

What do people think about keeping or dropping the automatic dimension coordinates that xarray creates?

Is there any way to know for sure whether a coordinate has been automatically created? Or is the name the only clue?

Latest commit fixes bug if the Iris cube does not have any dimension coordinates. What do people think about keeping or dropping the automatic dimension coordinates that xarray creates? And if dropping should be done should all coordinates named dim{x} be dropped? Or are we happy with just leaving them there?

For example at the moment:

``` python

print(cube) sea_water_potential_temperature / (degC) (-- : 148; -- : 180) Auxiliary coordinates: latitude x x longitude x x Scalar coordinates: depth: 4.99994 m, bound=(0.0, 10.0) m time: 1-01-01 12:00:00 Attributes: Conventions: CF-1.5 Cell methods: mean: time_counter print(xarray.DataArray.from_iris(cube).to_iris()) sea_water_potential_temperature / (degC) (dim0: 148; dim1: 180) Dimension coordinates: dim0 x - dim1 - x Auxiliary coordinates: latitude x x longitude x x Scalar coordinates: depth: 4.99993801117 m time: 1-01-01 12:00:00 Attributes: Conventions: CF-1.5

```

Ok, bounds can just be something to add to the converter if / when it's supported.

There's no good solution for representing bounds in xarray -- we can put these as coordinates on Dataset objects, but they aren't supported on DataArray objects. I had some plans to add an IntervalIndex type to pandas (https://github.com/pydata/pandas/pull/8707) which would let us support bounds, but honestly I'm not sure if/when I'll finish that.

There is a problem when it comes to bounds. Xarray adds the bounds as another DataArray to the DataSet, i.e.:

Data variables: votemper (dim0, dim1) float64 nan nan nan nan nan nan nan nan nan ... deptht_bnds (bnds) float64 0.0 10.0

I can get the name of this variable from the Coord attribute in the DataArray for another variable, i.e. ds['votemper'].coords['deptht'].bounds but can't get the bound array from the DataArray? is that correct?

Thanks for pinging @shoyer. I'll also ping @rhattersley, and one of us will take a good look in the next few days. :+1:

This is awesome!

Let's try to add some documentation -- at least a mention in What's New.

CC @pelson

docker pull nparley/xarray_test is a python 2.7 environment with Iris etc. which mirrors the travis CONDA_ENV=py27-cdat+iris+pynio.

It doesn't do everything, but it's a good first go. Here is an example:

``` python

import xarray import iris nc_file = 'E1_north_america.nc' cube = iris.load_cube(nc_file) darray = xarray.open_dataset(nc_file)['air_temperature'] print(darray) <xarray.DataArray 'air_temperature' (time: 240, latitude: 37, longitude: 49)> [435120 values with dtype=float32] Coordinates: * time (time) datetime64[ns] 1860-06-01 1861-06-01 ... * latitude (latitude) float32 15.0 16.25 17.5 18.75 20.0 ... * longitude (longitude) float32 225.0 226.875 228.75 ... forecast_period (time) timedelta64[ns] 449 days 18:00:00 ... forecast_reference_time datetime64[ns] 1859-09-01T06:00:00 height float64 1.5 Attributes: standard_name: air_temperature units: K Model scenario: E1 ukmo__um_stash_source: m01s03i236 source: Data from Met Office Unified Model 6.05 cell_methods: time: mean (interval: 6 hour) print(xarray.DataArray.from_iris(cube)) <xarray.DataArray u'air_temperature' (time: 240, latitude: 37, longitude: 49)> [435120 values with dtype=float32] Coordinates: * time (time) datetime64[ns] 1860-06-01 1861-06-01 ... * latitude (latitude) float32 15.0 16.25 17.5 18.75 20.0 ... * longitude (longitude) float32 225.0 226.875 228.75 ... forecast_reference_time datetime64[ns] 1859-09-01T06:00:00 height float64 1.5 forecast_period (time) timedelta64[ns] 449 days 18:00:00 ... Attributes: Model scenario: E1 STASH: m01s03i236 Conventions: CF-1.5 source: Data from Met Office Unified Model 6.05 standard_name: air_temperature units: K print(cube) air_temperature / (K) (time: 240; latitude: 37; longitude: 49) Dimension coordinates: time x - - latitude - x - longitude - - x Auxiliary coordinates: forecast_period x - - Scalar coordinates: forecast_reference_time: 1859-09-01 06:00:00 height: 1.5 m Attributes: Conventions: CF-1.5 Model scenario: E1 STASH: m01s03i236 source: Data from Met Office Unified Model 6.05 Cell methods: mean: time (6 hour print(darray.to_iris()) air_temperature / (K) (time: 240; latitude: 37; longitude: 49) Dimension coordinates: time x - - latitude - x - longitude - - x Auxiliary coordinates: forecast_period x - - Scalar coordinates: forecast_reference_time: 1859-09-01 06:00:00 height: 1.5 m Attributes: Model scenario: E1 source: Data from Met Office Unified Model 6.05 ukmo__um_stash_source: m01s03i236 ```

Reference https://github.com/pydata/xarray/issues/621 and https://github.com/pydata/xarray/issues/37. Uses template from https://github.com/pydata/xarray/pull/236.