issue_comments

With the current PR we would end up with two different build numbers with differing behaviour, which might confuse folks.

+1

But I'd rely on @ocefpaf's expertise.

The PR is a good idea and we, conda-forge, even though about making something like that for all packages. The problem is that optional packages metadata in Python-land is super unreliable. So, doing it on a package bases and with the original authors as part of it, it is super safe and recommended.

@dcherian do you mind taking a look at https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/pull/426? Please check the versions patched and the applied patch! Thanks!

We need to do a repodata patch for the current xarray. I'll get to it soon.

@kthyng those files are on a remote server and that may not be the segfault from the original issue here. It may be a server that is not happy with parallel access. Can you try that with local files?

PS: you can also try with netcdf4<1.6.1 and, if that also fails, it is most likely the server than the issue here.

that looks like it works, besides the fact that conda reports numbagg=0.2.1 but

It is not a conda problem but a bug upstream. The v0.2.1 tag hasv0.2.0 hardcoded in it.

See https://github.com/numbagg/numbagg/blob/v0.2.1/setup.py#L24

Note that this is not a bug per se, netcdf-c was never thread safe and, when the work around were removed in netcdf4-python, this issue surfaced. The right fix is to disable threads, like in my example above, or to wait for a netcdf-c release that is thread safe. I don't think the work around will be re-added in netcdf4-python.

This fix will restrict you to serial compute.

I was waiting for someone who do stuff on clusters to comment on that. Thanks! (My workflow is my own laptop only, so I'm quite limited on that front :smile:)

Also, you can try:

python import dask dask.config.set(scheduler="single-threaded")

That would ensure you don't use threads when reading with netcdf-c (netcdf4).

Edit: this is not an xarray problem and I recommend to close this issue and follow up with the one already opened upstream.

I believe you are hitting https://github.com/Unidata/netcdf4-python/issues/1192

The verdict is not out on that one yet. Your parallelization may not be thread safe, which makes 1.6.1 failures that expected. For now, if you can, downgrade to 1.6.0 or use an engine that is thread safe. Maybe h5netcdf (not sure!)?

Everything looks OK and I cannot reproduce that. Sorry, but I'm at a loss here.

The difference between your environment and mine does not explain it:

```

diff -Naur conda-list-mine.txt conda-list.txt ⏎ --- conda-list-mine.txt 2020-07-27 11:43:57.271609852 -0300 +++ conda-list.txt 2020-07-27 11:43:38.451218465 -0300 @@ -18,7 +18,7 @@ cartopy 0.18.0 py38h172510d_0 conda-forge certifi 2020.6.20 py38h32f6830_0 conda-forge cf-units 2.1.4 py38h8790de6_0 conda-forge -cffi 1.14.0 py38hd463f26_0 conda-forge +cffi 1.14.1 py38h5bae8af_0 conda-forge cfgrib 0.9.8.3 py_0 conda-forge cfitsio 3.470 hce51eda_6 conda-forge cftime 1.2.1 py38h8790de6_0 conda-forge @@ -77,7 +77,7 @@ kiwisolver 1.2.0 py38hbf85e49_0 conda-forge krb5 1.17.1 hfafb76e_1 conda-forge lcms2 2.11 hbd6801e_0 conda-forge -ld_impl_linux-64 2.34 h53a641e_7 conda-forge +ld_impl_linux-64 2.34 hc38a660_9 conda-forge libaec 1.0.4 he1b5a44_1 conda-forge libblas 3.8.0 17_openblas conda-forge libcblas 3.8.0 17_openblas conda-forge @@ -87,14 +87,14 @@ libffi 3.2.1 he1b5a44_1007 conda-forge libgcc-ng 9.2.0 h24d8f2e_2 conda-forge libgdal 3.0.4 he6a97d6_10 conda-forge -libgfortran-ng 7.5.0 hdf63c60_6 conda-forge +libgfortran-ng 7.5.0 hdf63c60_10 conda-forge libgomp 9.2.0 h24d8f2e_2 conda-forge libiconv 1.15 h516909a_1006 conda-forge libkml 1.3.0 hb574062_1011 conda-forge liblapack 3.8.0 17_openblas conda-forge libllvm9 9.0.1 he513fc3_1 conda-forge libnetcdf 4.7.4 nompi_h84807e1_105 conda-forge -libopenblas 0.3.10 pthreads_hb3c22a3_3 conda-forge +libopenblas 0.3.10 pthreads_hb3c22a3_4 conda-forge libpng 1.6.37 hed695b0_1 conda-forge libpq 12.3 h5513abc_0 conda-forge libsodium 1.0.17 h516909a_0 conda-forge @@ -220,4 +220,4 @@ zict 2.0.0 py_0 conda-forge zipp 3.1.0 py_0 conda-forge zlib 1.2.11 h516909a_1006 conda-forge -zstd 1.4.5 h6597ccf_1 conda-forge +zstd 1.4.5 h6597ccf_2 conda-forge ```

What is your shell? What is the result of echo $UDUNITS2_XML_PATH?

I could not reproduce it locally on my Linux box. Also, I believe the CIs here to the same, right? Maybe you could send me your .condarc and conda list so we can check if there is something off.

This is not needed. The pytest rc on the main channel was a mistake and it is already resolved. (Sorry about that BTW.)

I installed xarray through the recommended command in the official website in my minicoda env some months-year ago

That is probably it then. I see you have libnetcdf 4.6.2, if you recreate that env you should get libnetcdf 4.7.4. Can you try it with a new clean env:

shell conda create --name TEST --channel conda-forge xarray dask netCDF4 bottleneck

How are you installing netcdf4? There was a problem with the underlying libetcdf some time ago that caused access failures like that. You can try upgrading it or using another backend, like pydap.

@dcherian it is very experimental but nbrr can probably help you setup the environment.yaml file with a single command.

Thanks @cspencerjones . Can you add a test that tests this?

I guess we can just extend the existing test and check if we can find a coordinate by its attributes. For example, add a standard name for time here and see if we can find time using it.

The problem here is that we had to remove some old builds of gdal (a pynio dependency) due to an incompatibility with a newer icu. However, the new builds are using hdf5 1.10.5 but h5py is not ready for it yet. (h5py still uses hdf 1.10.4.)

@shoyer conda-forge dropped support for Python 3.5 a while back and the old packages may present a problem.

I agree erdappy seems like good fit for an xarray backend.

Note that erddapy is not really a "client" it is a URL builder that leverages ERDDAP's RESTful Web Services and pandas or xarray to download data.

@rmendels and @jhamman please correct me if I'm wrong below:

The main advantage of ERDDAP, as @rmendels mentioned above, is that one can make requests in coordinate space. However, I don't think there is an OPeNDAP response for that a sliced request in ERDDAP, just the "full data." Also, xarray can already ingest an OPeNDAP URL and lazily slice it using high level coordinate space syntax. (At the cost of loading the coordinates but that is 99% of the time fast enough.)

The alternative would be to use the netCDF response but that means we would need to download the file and load it with xarray, which does not really justify a new backend IMO. (I do that in a very inelegant way in erddapy.)

The download limitation exists in any of the "file format" responses in ERDDAP, making the OPeNDAP the best choice. So, unless we can figure out a way for ERDDAP to serve the "sliced" URL as OPeNDAP, I believe the best we can do already exists in xarray :grimacing:

h10edf3e_1 contains the timeout fix and is build against hdf5 1.10.2. The conda-forge h9cd6fdc_11 build is against hdf5 1.10.3 perhaps that makes a different?

There are many variables at play here. The env that solved it in https://github.com/pydata/xarray/issues/2503#issuecomment-432645477 seems quite different from the env where the problem happened, including an xarray dev version. I'm not sure hdf5 is a good candidate to blame :smile:

In defaults libnetcdf4 4.6.1 build 1 and above contain the timeout fix, build 0 has the original timeout.

Thanks @jjhelmus! I guess that info and https://github.com/pydata/xarray/issues/2503#issuecomment-432483817 eliminates the timeout issue from the equation.

That version has the fix for the issue.

I know that @jjhelmus ported the fix to defaults but I'm not sure which build number has it, and/or if the previous one was remove, b/c defaults builds are not as transparent as conda-forge's :smile:

He can probably say more about that.

@shoyer and @jhamman this looks good to go IMO. @DocOtak thanks for fixing my bug!

I believe the last one in the notebook below is already fixed and the first two are mentioned above but here is a data point:

http://nbviewer.jupyter.org/gist/ocefpaf/1bf3b86359c459c89d44a81d3129f967

I can work on a PR tomorrow. Does the benefit of having the same behavior as the netCDF4 library warrant a potentially breaking change for existing code which relies on the current behavior of filter_by_attrs()?

IMO, yes.

This might need adding a new method with the same behavior as netCDF4 and keeping the existing one as is (with appropriate documentation updates).

That is up to xarray devs but I personally don't think it is necessary.

Got it. I cannot dig into this at the moment but having both implementations working in a consistent way would be nice. Do you want to send a PR?

it appears that this scenario was not directly contemplated.

Correct. I did not foresee that use case. Are you sure that netCDF4.Dataset.get_variables_by_attributes behaves as a logical AND? The code is virtually a copy-n-paste from there.

Notice that it took 411 seconds to run conda env create!

If you are using conda-forge bare in mind that our package index is huge and conda is not very smart about it. We are looking into possible solution. Pinging @pelson who have some ideas in mind on how to address this problem.

It is not ideal but you can workaround that by dropping the siglay variable.

python ds = xr.open_dataset(url, drop_variables='siglay')

I would like to be able to round-trip something like the following dataset to netCDF.

OK, I though you meant roundtrip from the netCDF file and back. In my line of work handling high level python objects serialization like that is usually not desired as the user should be responsible of how s/he wants to save the data. (I also never had an application that required timedelta, so it was hard for me to contextualize that.)

I'll be away the next weeks, so if someone wants to pick it up from here please feel free to do so. Otherwise I'll try to get back to this once I return to the office.

@shoyer this is ready for a second round of reviews.

The last thing I'd like to see is support for automatically serializing/deserializing timedelta64 by saving an attribute dtype='timedelta64[ns]'. This will preserve the ability to roundtrip this data to netCDF, which I do think is useful for some users -- otherwise users will have xarray objects that they can no longer save to netCDF.

I'm not sure I follow. Roundtrip is easier now since the original time unit dtype is preserved, no? Ping @rsignell-usgs who is a netCDF/CF specialist :wink:

Thanks! I'll look into those and should have something by next week.

@shoyer what is the path forward? In https://github.com/pydata/xarray/pull/940 I implemented a keyword so we could keep both behaviors, which I believe is a bad idea.

Would a PR changing the current behavior and return floats instead timedelta64 be OK? If so I can try to put that together over this weekend.

@spencerkclark conda-forge has netcdf4 built with both 4.5.0 and 4.4.1.1, so conda can get one or the other depending on the scenario. The only way to ensure you are building an env with libnetcdf 4.5.0 is to add it explicitly in that list.

I removed the bad package and things should be back t normal. See https://anaconda.org/conda-forge/hypothesis/files?version=3.33.0

(Not sure what went wrong and I cannot look into it right now, but I guess that latest hypothesis is not crucial for you here.)

I have never found timedelta64 indices to be particularly useful.

Same here. :+1: for 1

PS: 2 could be the start of a nice "CF-addon" package for xarray but I don't think it should be in the xarray code.

Yep, just saw that in https://github.com/conda-forge/pynio-feedstock/pull/30

So we need to wait... Hopefully they will release it soon. Thanks!

Do you know if the py3k support made into pynio 1.5.0 release? If so I can "fix" the conda package easily, if not I am unsure what source I should use to enable the Python 3 builds.

Totally missed your answer here @shoyer. Thanks!

The workaround is fine and the _FillValue =-1 seems wrong to me. Pinging @rsignell-usgs who is know more about the conventions and was interested into this in the first place.

Closing this as I don't think anything is broken with xarray.

@JiaweiZhuang let's discuss that in the feedstock issue tracker to avoid cluttering xarray's.

then some effort needs to be made to build conda recipes and other infrastructure for distributing and building the platform.

Like https://github.com/conda-forge/esmf-feedstock :wink:

(Windows is still a problem b/c of the Fortran compiler.)

Something is not OK when parsing ocean_time (and some other variables).

If you do d['ocean_time'][:] on your example with netCDF4 you'll get the same error as xarray.

Could it be a bad aggregation on the THREDDS service?

Ignore what I said. looking closer it makes sense to set the default to True.

Would it make sense to set the default to False? I believe false is used in mpl.

TL;DR you may consider making this change permanent.

@fmaussion the problem is that latest conda-forge's hdf4 does ship with libmfhdf.so.0 but either defaults' version does not or an old version in conda-forge is broken. (I am traveling but I'll confirm that as soon as I get back.)

Adding hdf4 at the env creation forces the solver to get the latest version/build number and prevented the downgrade in the subsequent conda install call. So you should probably leave hdf4 there even if it is over specifying deps because the issue may resurface.

I say that b/c, if the bad version is in conda-forge we can remove it. But if it is in defaults we have no control.

@ocefpaf where you able to make any progress on this?

Sorry but no. I will look at it again but I do remember seeing some extra conda install commands issued after the env was create, and that may lead to undesirable up-downgrade of packages. If possible, the RTD env should install everything in one go, with a single conda env create environment.yml.

PS: Not sure what your timezone is but due to my ignorance on RTD it would be nice if we could touch base on gitter at some point. We can probably figure this out quickly.

We are talking about two different issues here. I'll open another issue for the rasm file.

Ah OK. Sorry for the noise.

The error is visible here the latest build logs are available here thanks!

Thanks @fmaussion I can see that there are multiple conda install commands the the subsequent ones change the env. For example:

``` conda install --yes --name latest sphinx==1.3.5 Pygments==2.1.1 docutils==0.12 mock pillow==3.0.0 sphinx_rtd_theme==0.1.7 alabaster>=0.7,<0.8,!=0.7.5

Fetching package metadata: .... Solving package specifications: .........

Package plan for installation in environment /home/docs/checkouts/readthedocs.org/user_builds/xray/conda/latest:

The following NEW packages will be INSTALLED:

jbig:             2.1-0       
sphinx_rtd_theme: 0.1.7-py27_0

The following packages will be DOWNGRADED:

freetype:         2.6.3-1      --> 2.5.5-1     
jpeg:             9b-0         --> 8d-2        
libtiff:          4.0.6-7      --> 4.0.6-2     
pillow:           3.4.2-py27_0 --> 3.0.0-py27_1
pygments:         2.1.3-py27_1 --> 2.1.1-py27_0
sphinx:           1.4.8-py27_0 --> 1.3.5-py27_0
tk:               8.5.19-0     --> 8.5.18-0

```

That jpeg change is not desirable!

I also cannot see the full log from conda env create --name latest --file /home/docs/checkouts/readthedocs.org/user_builds/xray/checkouts/latest/doc/environment.yml.

It's the same environment, I'm literally typing "git checkout master" or "git checkout v0.8.2" in my xarray-dev conda environment (on Python 3.5)

I am not sure it is due to the change I mention above. I will make a few experiments and report back.

Can you point me to the error and some details on RTD. They may need to update the conda version to get it to work.

@fmaussion I am away from a laptop to test this but the following change should fix it (depending on the conda version you have there) :

yaml channels: - conda-forge - defaults

By explicitly adding defaults below conda-forge you will ensure that the right set of packages will be downloaded thanks to the channel preference feature in recent conda version.

We definitely ignore cell boundaries -- they don't (yet) have any place in the xarray data model.

Even though I would love to have that functionality I do not believe it is high priority. cell boundaries is a corner case, at least in my field of work, and 99% of the time it is OK to infer the intervals. Also, the name of the key word is quite clear: infer_intervals.

Maybe you could only add a note in the docs mentioning that the cell boundaries might exist?

Yes, it is a special case for pcolormesh though

A special case that I love. In the old Matlab day we had horrible hacks to get pcolor to work properly :smile:

I'm not sure if the additional complexity added by cell boundaries is on the xarray devs priority list...

That is fine. Maybe a warning in the docs would be nice. So people can revert to a "manual" plotting to get the boundaries right.

Is infer_intervals creating coordinate bounds for plotting? It looks like it. What about when we do have cell boundaries in the dataset? (See http://cfconventions.org/cf-conventions/v1.6.0/cf-conventions.html#cell-boundaries). Will those be ignored and the inferred one used?

(Sorry if I am only making noise here and this does not make sense. In that case just ignore my comment.)

which version couples to who was not obvious

Sorry, this is an old issue that I had in the back of my head but I do not remember the versions :grimacing: but a general update should get you going.

in the end what worked was installing cartopy from conda-forge repository.

:+1:

That is unrelated to xarray. It is a cartopy/shapely bug due to version mismatch. If you update both that should work.

@shoyer the more I think about this the more I don't like the addition of extra keywords. Even though I would like this behavior to be the default one I really do not like the complexity I added here.

Closing this...

@shoyer the more I think about this the more I don't like the addition of extra keywords. Even though I would like this behavior to be the default one I really do not like the complexity I added here.

Closing this...

If the main issue is plotting, you could try fixing that upstream, too! pydata/pandas#8711

Indeed! That makes sense to matter what is decided here. Thanks for pointing that out. (Not sure if I am up to the challenge though.)

It might be worth querying the mailing list for more opinions here.

Done!

Decoding time units into timedelta64 is useful if you want to be able to use them for arithmetic with datetimes.

I get that but my question is how often do users perform do such operations? Again I am biased b/c with my data I never want to do that as it does not make sense. And, when it does make sense, I believe that the price of post conversion is worth the advantages of converting by default.

Still I'm somewhat reluctant to change the default here.

If you want decode_timedeltas=True as the default I am not sure it is worth the extra keyword then. My goal with this PR is to avoid extra steps. If users need to set it to True it is not too different from doing data.values / 1e9. If all we get is to change one step for another I prefer to not make the open_dataset more complex and leave it as is.

Feel free to close this. I don't have strong feelings about what xarray should do by default. I just want to make it more convenient for my uses cases.

This is ready for review.

Here is an example of this PR in action showing how plotting and working with periods gets easier with floats instead of timedeltas.

BTW, in light of https://github.com/pydata/xarray/issues/939#issue-169274464, I wonder if decode_timedeltas and decode_datetimes (or even the original decode_times) are needed at all. (Just defending myself as I don't really want a new keyword argument but a better default for time data in general :grimacing:)

Maybe I am being thick and I don't know enough use cases data but I cannot see why someone might want to convert time data (which most of the time represent periods) to timedeltas. That breaks from the original data units and forces the user to compute it back and convert the dtype too before for plotting, etc.

Regarding time coordinate itself I understand that xarray, due to the pandas nature of the index, needs to fail to convert time when the calendar is not supported by pandas. Maybe, instead of throwing an error and asking the users to use the option decode_(date)times=False in case of failure, xarray could issue an warning and return only the "numbers" as if decode_(date)times were set to False.

I understand, and agree most of the time, that raising erros is better than issuing warnings, and creating an ambiguity in the returns. So maybe this one is harder to change than the former.

Prototype is ready http://nbviewer.jupyter.org/gist/ocefpaf/cabdcb27a5ef7da1b6110327a5f03e17

I will polish this and send a PR soon.

No problem. I want to solve this anyways b/c it is quite awkward to keep converting back from timedeltas to time data. I will have a PR soon.

@shoyer is this what you have in mid? Replacing decode_time for decode_datetimes and decode_timedelta and then:

``` python if decode_datetimes and 'units' in attributes and 'since' in attributes['units']: units = pop_to(attributes, encoding, 'units') calendar = pop_to(attributes, encoding, 'calendar') data = DecodedCFDatetimeArray(data, units, calendar)

if decode_timedelta and attributes['units'] in TIME_UNITS:
    units = pop_to(attributes, encoding, 'units')
    data = DecodedCFTimedeltaArray(data, units)

```

To be honest I cannot see why someone might want to convert time data to a timedelta and I would be inclined to remove decode_timedelta instead. However, that may be my very biased view since such transformation does not make sense with my data :smile:

Do you have any advice? (I would like to try this before you release v0.8.0)

Done. Not sure why only AppVeyor started :confused:

Assuming we do add the filter method, maybe filter_by_attrs?

Makes sense. I will modify this soon.

I do think the name is a mouthful, though :).

I agree that is a mouthful but then again, I was trying to be consistent with the existing versions. However, I don't really have a strong opinion on the matter and I am fine with whatever you decide. Should I renamed then? filter_attrs? (Sounds weird though b/c you are not filtering the attrs, but the variables based on the attrs :expressionless:)

I fine with whatever you decide but here are my two cents: - get_variables_by_attributes is the same name of this method in netcd4 and some java netcdf libraries. So I'd rather not have a specialized version for attributes than adding it with name with a different name. - I see the elegance in s.filter(lambda x: x.attrs['standard_name'] == 'convective_precipitation_flux') and I like it a lot! But the specialized version for attributes is more compact to write and, at least in my field, filtering by attributes is more common making this version more convenient.

Feel free to close this if you don't think it is worth adding the specialized version. Or let me know if you want to rename it.

Is there still an interested in this or should I close in light of https://github.com/pydata/xarray/issues/883?

Agreed -- this is almost ready. Please add to the API docs (api.rst) and do the docstring fixes.

Not sure if I did this right :grimacing:

Rebased and ready for another round of reviews :wink:

@ocefpaf Looks like it's the official Continuum build of SciPy that just started failing here (entirely coincidentally). I only switched to conda forge for our build with all optional dependencies py27-cdat+pynio. So I'm going to merge this for now and fix the minimal build later.

Cool. I will take a look at adding coveralls once I get back in office. (Heading to PyCon. Will you be there @shoyer?)

@shoyer I am interested in help you investigate what is going on here as that will help us to keep the conda-forge stable. Can you please add conda config --set show_channel_urls true in the .travis.yml so we know where the packages are coming from? We just added scipy to conda-forge and I am afraid that might be the issue here.

Ping @jakirkham who added the scipy recipe to help investigate the issue :grimacing:

Sorry, I'm going camping this weekend so I won't be able to get this out. Next time give me just a little bit more warning :).

No biggie. As I mentioned above I have a plan B (conda install of the latest dev version).

Enjoy your camping!

The appveyor build failure looks unrelated to this change -- something about conda dependencies.

I am experiencing that in other projects. It is actually a bad download of miniconda, and powershell makes the error message unusable. Re-starting should fix it. (BTW miniconda is pre-installed on AppVeyor and that download is not needed.)

@shoyer I will be teaching a tutorial next Monday that will hit this bug. I know it is a lot to ask... But do you think you could cut a bugfix release? (I have a plan B be already, so no pressure.)

@jhamman and @shoyer if the tests passes this is ready for another round of review. (Let me know if I should squash the previous ones to make it easier to review.)

@shoyer this is ready for another round of review.

Well, if you guys are mostly excited about using this for coordinate variables, another consistent choice would be to return a list of matching DataArrays. But if we want to return a Dataset, we should only do data variables, because it's weird to lose all the describing coordinates when you match, e.g., standard_name="air_temperature".

We agree with you and I prefer to return a Dataset.

Right now we always go to netCDF4-python to do this low-level CF interpretation stuff. If we start using xarray for that task we should improve xarray, to take advantage of the conventions (CF/SGRID/UGRID), instead of using xarray to just find the coords data. (For example: creating the z coordinates from non-dimension coordinates and add that to the Dataset coords automatically.)

After discussing with my CF guru (@rsignell-usgs) and the original author of the get_variables_by_attributes (@kwilcox) I jumped the fence and I am OK leaving the xarray model pure and filtering only the data variables.

should this filter both data variables and coordinates or only data variables?

I thought a little bit more about this and now I am on the fence. The pros of filtering only data variables are a nice and clean Dataset object, and overall consistency with the high level xarray model. The cons are that we cannot do the filtering on the coords (obviously), but most of the time that we need to do that we go to a lower level object like netCDF4.Dataset. However, it would be nice if both xarray and netCDF4 behaved the same way...

I am 51% with filtering both (and the current implementation does that) but I will leave the final decision to you.

An important design question: should this filter both data variables and coordinates or only data variables? My thought is that it's only worth filtering data variables -- filtering out unmatched coordinates is not very useful.

I understand that returning coordinates without a data variable associated to them seems weird to the high level model of xarray.Dataset , but I disagree that it is not very useful. In fact that is the most common operation I do: find coordinates based on axis, formula_terms, etc and construct common grids for plotting and/or building the z coords from non-dimension coords.

possibly we should add an explicit toggle for decoding timedeltas vs datetimes.

:+1:

I am opening a separated issue for that.

@shoyer this made into netcdf4 and some people in my group would like to have this in xarray too. If you think it is worth it I can put a PR together for this.

Looks great! Please add a brief bug fix note to "What's new", then I will merge.

How about https://github.com/pydata/xarray/pull/842/commits/518ea53284f659edbb31cd98c326b3e78f440fc3?

PS: @shoyer I am still not sure that converting any data that has units of time to timedelta is desirable as the default behavior. I may be biased but in my datasets (waves period data) we usually do not want that.

second = first.reindex_like(second, method='nearest', tolerance=0.001)

I really like this. Explicit and self-documenting code. I would avoid making this automatic.

What if there are bounds in the file and the data is regularly spaced? I consider the current behavior a guess, and guessing should be an active user choice, not the automatic behavior.

Bare in mind that some netCDF4 files do have the bounds data and those should be used as "breaks" when available.

BTW I'd rather not have the _infer_interval_breaks at all, not sure if assuming linear at the mid points are generic enough.

I don't think it's too much to ask for one more py33 build after the rename.

Yes and no. If you want the full optional dependencies, xarray can be a little bit hard to build (with pynio support, etc). If you want just the basics then it is relatively easy.

You should make the conda package request in https://github.com/ContinuumIO/anaconda-issues/issues/635, and other continuum communication channels (mailing list, gitter, etc).

The use of "official release" can be confusing in this case. I believe that the latest xarray works OK in Python 3.3. But note that the project is not tested on Python 3.3 since PR https://github.com/pydata/xarray/pull/583. I believe that the official PyPI source dist will install OK in Python 3.3 BTW.

On the other hand, Continuum does seem to be abandoning Python 3.3, there are hardly new conda packages being built for it.

@richardotis any special reason for wanting a Python 3.3 version? The migration to >=3.4 is highly recommended. (And I don't think continuum and the community building stuff for Python 3.3 lately.)

If you cannot upgrade you can use this recipe to build you own Python 3.3 xarray.

If you clone the whole repo you can then type:

shell export CONDA_PY=33 export CONDA_NPY=110 # (or any other numpy version series you need 18, 19...) conda build xarray

If you issue the last command from the repo root directory conda will build dependencies for you.

conda install -c ioos xarray should work. ;-)

:-1:

But I am just one user... And if the renaming really happens I guess that import xarray as xr is better than xry

(I like the pun: See the data under x rays :smile:)

Got it. Thanks!

I guess I've been living around numpy.arrays way too long... Time to experiment this brave new world of labeled arrays :wink:

Hi @rabernat,

Most gsw functions will call np.broadcast_arrays for you internally. So you can pass ds.a.values instead. It is ugly, I know. but consistent when using libraries that expect numpy array.

but it would be nice to be able to say that you use a standard license

:+1:

The license here: http://www.pyngl.ucar.edu/Licenses/PyNIO_source_license.shtml

@david-ian-brown sorry for my ignorance but is that OSI approved?

Thanks @david-ian-brown! That is good news!!

Maybe we could get ioos to also keep a pynio conda build up to date (cc @ocefpaf)

@david-ian-brown is pynio open source now? If so I can add it to the ioos channel, but at PyPI it is still missing the source.

single option for open_dataset that can disable all of xray's decoding options

I get that and I really like this option!

Not sure about raw=False

:+1: decode=True