seems like for, e.g., head, you can pass either a single dimension or multiple ones (e.g., either as **kwargs or a dictionary) and use those as the start dimension.

that said, about naming conventions, i think for tensors the most common convention is definitely slice() (which is implemented as isel). head/tail can be implemented in terms of slice().

e.g.: ds.slice(dim1=3, dim2=(1,4), dim3=(1,None,5)) -- or -- ds.slice({'dim1': 3, 'dim2': (1,4), 'dim3': (1, None, 5)})

head/tail/whatever are easy calls to this and you can have that in the documentation. as a result, people won't get confused because they understand slice.

On Tue, Feb 10, 2015 at 4:14 PM, Stephan Hoyer notifications@github.com wrote:

@ebrevdo https://github.com/ebrevdo yes, I think we probably need to stick to the pandas/numpy convention for the meaning of take.

My inspiration for these names was the head() and tail() methods in pandas, which are quite convenient. But it's not entirely clear how/if these generalize cleanly to N-dimensions. I suppose take_first and take_last could be an improvement over head/tail.

— Reply to this email directly or view it on GitHub https://github.com/xray/xray/issues/319#issuecomment-73812481.

Clojure conventions: .take, .take_last get the first n and last n pandas/ndarray conventions: .take([3,4,5]) selects rows 3,4,5.

probably want to be consistent with one of these.

On Tue, Feb 10, 2015 at 3:28 PM, Stephan Hoyer notifications@github.com wrote:

These would be shortcuts for isel/slice syntax: - ds.head(time=5) -> ds.isel(time=slice(5)): select the first five time values - ds.tail(time=5) -> ds.isel(time=slice(-5, None)): select the last five time values - ds.subsample(time=5) -> ds.isel(time=slice(None, None, 5)): select every 5th time value

— Reply to this email directly or view it on GitHub https://github.com/xray/xray/issues/319.

Why not options like 'first', 'common', etc? On May 20, 2014 11:52 PM, "akleeman" notifications@github.com wrote:

Yeah I agree, seems like a great option to have.

— Reply to this email directly or view it on GitHubhttps://github.com/xray/xray/issues/138#issuecomment-43718217 .

I definitely like the inplace idea. We could also use the function name update in this case. On Mar 12, 2014 9:49 AM, "Stephan Hoyer" notifications@github.com wrote:

I'm thinking now that we should move Dataset.replace into Dataset.set_variables, with the keyword argument "inplace" (like Dataset.merge). This would consolidate the public interface (which is usually a good thing).

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/pull/62#issuecomment-37432745 .

True. Maybe stick with replace, and we can put filter on the to do? I may work on it tomorrow. On Mar 11, 2014 10:10 PM, "Stephan Hoyer" notifications@github.com wrote:

Yes, I suppose update would be an appropriate name -- except update strongly conveys (to me) that the operation is done in place (like how it works for Python dicts).

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/pull/62#issuecomment-37376178 .

Don't dicts have an update function that works this way? On Mar 11, 2014 9:48 PM, "Stephan Hoyer" notifications@github.com wrote:

Perhaps this method should be called "modify" instead of "replace", since it also lets you set or remove variables. Thoughts?

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/pull/62#issuecomment-37375305 .

This in response to your first bullet:

Create a new dataset based on some (but not all) variables from an existing dataset.

for that there's a filter() in pandas. it would be useful to have here as well.

On Tue, Mar 11, 2014 at 4:11 PM, Stephan Hoyer notifications@github.comwrote:

Not really. A filter method would be a useful addition for xray, but Dataset.replace is replacing entire variables, not values in variable.

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/pull/62#issuecomment-37358060 .

Is part 1 similar to the pandas .filter operator? That one has nice keywords, 'like', 'regex', etc.

Indices also have an .inferred_type getter. unfortunately it doesn't seem to return true type names...

In [13]: pandas.Index([1,2,3]).inferred_type Out[13]: 'integer'

In [14]: pandas.Index([1,2,3.5]).inferred_type Out[14]: 'mixed-integer-float'

In [15]: pandas.Index(["ab","cd"]).inferred_type Out[15]: 'string'

In [16]: pandas.Index(["ab","cd",3]).inferred_type Out[16]: 'mixed-integer'

On Sun, Mar 2, 2014 at 10:14 PM, Stephan Hoyer notifications@github.comwrote:

This is because coordinates are loaded as pandas.Index objects... which don't always faithfully preserve the type of the underlying object (see pydata/pandas#6471 https://github.com/pydata/pandas/issues/6471).

I believe serialization should still work though thanks to a work around I added for dtype=object. Do let me know if this is not the case. One solution to make this less awkward would be to wrap pandas.Index in something that keeps track of the dtype of the original arguments for use in mathematical expression.

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/issues/39#issuecomment-36484122 .

Some of these are specific to the datastore. nc3/nc4 may care about integer packing and masking, but grib format may not. maybe that's where these things should really reside. as aspects of the datastore object. not sure about units though. either way, ideally these would be transparent to the user of the xarray/dataset objects, except as parameters when reading/writing?

Agreed. I would avoid that kind of thing too. Maybe a stern warning for all conflicting attributes, and saying that they will be dropped from the new variable.

For units specifically, Python has a variety of unit libraries that wrap numpy arrays and can probably do some magic. Not sure if we really want to do that, though.

On Wed, Feb 26, 2014 at 4:07 PM, Stephan Hoyer notifications@github.comwrote:

Stern warnings about conflicting attributes like units may be an appropriate compromise. But if we go that way, I would advocate for making units drop upon doing any mathematical operation. We could try to update units automatically (e.g., kg * kg = kg^2), but that is tricky to always right.

Celsius, for example, is a pretty weird physical unit because of how it can take on negative values, so it actually makes a lot of sense to use mostly use Kelvin instead (for which I can sensibly apply any math operation like times or minus). That doesn't mean that I want to store all my raw data in degrees K, though...

On Wed, Feb 26, 2014 at 3:45 PM, ebrevdo notifications@github.com wrote:

err, which attributes conflict.

On Wed, Feb 26, 2014 at 3:45 PM, Eugene Brevdo ebrevdo@gmail.com wrote:

I don't think that example has your intended affect. I don't know why anyone would add something of units kelvin with those of celsius. I understand what you're saying, so maybe we should just throw a stern warning listing which units conflict and how, every single time.

On Wed, Feb 26, 2014 at 3:42 PM, Stephan Hoyer < notifications@github.com wrote:

I see your point, but I favor a more pragmatic approach by default. See my fourth bullet under "Design Goals" in the README and bullet ii under Iris in "Prior Art".

My vision here is a more powerful ndarray enhanced rather than limited by metadata. This is closer to what pandas does, which even allows for conflicting indices resulting in NaN values (a feature I would love to copy).

I think that both use cases can be covered as long as the merge/conflict logic is clearly documented and it is possible to write stricter logic for library code (which by necessity will be more verbose). If it is essential for units to agree before doing x + y, you can add assert x.attribubes.get('units') == y.attributes.get('units'). Otherwise, we will end up prohibiting operations like that when x has units of Celsius and y has units of Kelvin.

On Wed, Feb 26, 2014 at 3:23 PM, ebrevdo notifications@github.com wrote:

Also, there are plenty of other bits where you don't want conflicts. Imagine that you have variables indexed on different basemap projections. Creating exceptions to the rule seems like a bit of a rabbit hole.

On Wed, Feb 26, 2014 at 3:13 PM, Eugene Brevdo ebrevdo@gmail.com wrote:

This is an option, but these lists will break if we try to express other data formats using these conventions. For example, grib likely has other conventions. We would have to overload attribute or variable depending on what the underlying datastore is.

On Wed, Feb 26, 2014 at 3:03 PM, Stephan Hoyer < notifications@github.com wrote:

x + y could indeed check variable attributes before trying to do the merge. I don't know if it does in the current implementation.

My concern is more that metadata like "title" or "source" should not be required to match, because that metadata will almost always be conflicting. Perhaps "units", "_FIllValue", "scale_factor" and "add_offset" (if

values were not automatically masked/scaled) should be specifically blacklisted to prohibit conflicts.

Reply to this email directly or view it on GitHub< https://github.com/akleeman/xray/issues/25#issuecomment-36189171> .

Reply to this email directly or view it on GitHub< https://github.com/akleeman/xray/issues/25#issuecomment-36190935>

.

Reply to this email directly or view it on GitHub< https://github.com/akleeman/xray/issues/25#issuecomment-36192859> .

Reply to this email directly or view it on GitHub< https://github.com/akleeman/xray/issues/25#issuecomment-36193148>

.

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/issues/25#issuecomment-36194729 .

err, which attributes conflict.

On Wed, Feb 26, 2014 at 3:45 PM, Eugene Brevdo ebrevdo@gmail.com wrote:

I don't think that example has your intended affect. I don't know why anyone would add something of units kelvin with those of celsius. I understand what you're saying, so maybe we should just throw a stern warning listing which units conflict and how, every single time.

On Wed, Feb 26, 2014 at 3:42 PM, Stephan Hoyer notifications@github.comwrote:

I see your point, but I favor a more pragmatic approach by default. See my fourth bullet under "Design Goals" in the README and bullet ii under Iris in "Prior Art".

My vision here is a more powerful ndarray enhanced rather than limited by metadata. This is closer to what pandas does, which even allows for conflicting indices resulting in NaN values (a feature I would love to copy).

I think that both use cases can be covered as long as the merge/conflict logic is clearly documented and it is possible to write stricter logic for library code (which by necessity will be more verbose). If it is essential for units to agree before doing x + y, you can add assert x.attribubes.get('units') == y.attributes.get('units'). Otherwise, we will end up prohibiting operations like that when x has units of Celsius and y has units of Kelvin.

On Wed, Feb 26, 2014 at 3:23 PM, ebrevdo notifications@github.com wrote:

Also, there are plenty of other bits where you don't want conflicts. Imagine that you have variables indexed on different basemap projections. Creating exceptions to the rule seems like a bit of a rabbit hole.

On Wed, Feb 26, 2014 at 3:13 PM, Eugene Brevdo ebrevdo@gmail.com wrote:

This is an option, but these lists will break if we try to express other data formats using these conventions. For example, grib likely has other conventions. We would have to overload attribute or variable depending on what the underlying datastore is.

On Wed, Feb 26, 2014 at 3:03 PM, Stephan Hoyer < notifications@github.com wrote:

x + y could indeed check variable attributes before trying to do the merge. I don't know if it does in the current implementation.

My concern is more that metadata like "title" or "source" should not be required to match, because that metadata will almost always be conflicting. Perhaps "units", "_FIllValue", "scale_factor" and "add_offset" (if

values were not automatically masked/scaled) should be specifically blacklisted to prohibit conflicts.

Reply to this email directly or view it on GitHub< https://github.com/akleeman/xray/issues/25#issuecomment-36189171> .

Reply to this email directly or view it on GitHub< https://github.com/akleeman/xray/issues/25#issuecomment-36190935>

.

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/issues/25#issuecomment-36192859 .

I don't think that example has your intended affect. I don't know why anyone would add something of units kelvin with those of celsius. I understand what you're saying, so maybe we should just throw a stern warning listing which units conflict and how, every single time.

On Wed, Feb 26, 2014 at 3:42 PM, Stephan Hoyer notifications@github.comwrote:

I see your point, but I favor a more pragmatic approach by default. See my fourth bullet under "Design Goals" in the README and bullet ii under Iris in "Prior Art".

My vision here is a more powerful ndarray enhanced rather than limited by metadata. This is closer to what pandas does, which even allows for conflicting indices resulting in NaN values (a feature I would love to copy).

I think that both use cases can be covered as long as the merge/conflict logic is clearly documented and it is possible to write stricter logic for library code (which by necessity will be more verbose). If it is essential for units to agree before doing x + y, you can add assert x.attribubes.get('units') == y.attributes.get('units'). Otherwise, we will end up prohibiting operations like that when x has units of Celsius and y has units of Kelvin.

On Wed, Feb 26, 2014 at 3:23 PM, ebrevdo notifications@github.com wrote:

Also, there are plenty of other bits where you don't want conflicts. Imagine that you have variables indexed on different basemap projections. Creating exceptions to the rule seems like a bit of a rabbit hole.

On Wed, Feb 26, 2014 at 3:13 PM, Eugene Brevdo ebrevdo@gmail.com wrote:

This is an option, but these lists will break if we try to express other data formats using these conventions. For example, grib likely has other conventions. We would have to overload attribute or variable depending on what the underlying datastore is.

On Wed, Feb 26, 2014 at 3:03 PM, Stephan Hoyer < notifications@github.com wrote:

x + y could indeed check variable attributes before trying to do the merge. I don't know if it does in the current implementation.

My concern is more that metadata like "title" or "source" should not be required to match, because that metadata will almost always be conflicting. Perhaps "units", "_FIllValue", "scale_factor" and "add_offset" (if

values were not automatically masked/scaled) should be specifically blacklisted to prohibit conflicts.

Reply to this email directly or view it on GitHub< https://github.com/akleeman/xray/issues/25#issuecomment-36189171> .

Reply to this email directly or view it on GitHub< https://github.com/akleeman/xray/issues/25#issuecomment-36190935>

.

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/issues/25#issuecomment-36192859 .

Also, there are plenty of other bits where you don't want conflicts. Imagine that you have variables indexed on different basemap projections. Creating exceptions to the rule seems like a bit of a rabbit hole.

On Wed, Feb 26, 2014 at 3:13 PM, Eugene Brevdo ebrevdo@gmail.com wrote:

This is an option, but these lists will break if we try to express other data formats using these conventions. For example, grib likely has other conventions. We would have to overload attribute or variable depending on what the underlying datastore is.

On Wed, Feb 26, 2014 at 3:03 PM, Stephan Hoyer notifications@github.comwrote:

x + y could indeed check variable attributes before trying to do the merge. I don't know if it does in the current implementation.

My concern is more that metadata like "title" or "source" should not be required to match, because that metadata will almost always be conflicting. Perhaps "units", "_FIllValue", "scale_factor" and "add_offset" (if values were not automatically masked/scaled) should be specifically blacklisted to prohibit conflicts.

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/issues/25#issuecomment-36189171 .

This is an option, but these lists will break if we try to express other data formats using these conventions. For example, grib likely has other conventions. We would have to overload attribute or variable depending on what the underlying datastore is.

On Wed, Feb 26, 2014 at 3:03 PM, Stephan Hoyer notifications@github.comwrote:

x + y could indeed check variable attributes before trying to do the merge. I don't know if it does in the current implementation.

My concern is more that metadata like "title" or "source" should not be required to match, because that metadata will almost always be conflicting. Perhaps "units", "_FIllValue", "scale_factor" and "add_offset" (if values were not automatically masked/scaled) should be specifically blacklisted to prohibit conflicts.

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/issues/25#issuecomment-36189171 .

It depends on whether x+y does attribute checking before performing the merge. Again, if units don't match then maybe you shouldn't add. I always favor the strictest approach so you don't get strange surprises.

On Wed, Feb 26, 2014 at 2:47 PM, Stephan Hoyer notifications@github.comwrote:

Dataset.merge is also triggered by assigning a DatasetArray to a dataset or by doing a mathematical operation on two DatasetArrays (e.g., x + y). The later is how I encountered this issue today.

For merge itself, I would agree that we may want to default to stricter behavior, but for these other versions of merge we should default to something more flexible.

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/issues/25#issuecomment-36187723 .

``` In [73]: h = cPickle.dumps(f)

TypeError Traceback (most recent call last) <ipython-input-73-a875f963f7d4> in <module>() ----> 1 h = cPickle.dumps(f)

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy_reg.pyc in _reduce_ex(self, proto) 68 else: 69 if base is self.class: ---> 70 raise TypeError, "can't pickle %s objects" % base.name 71 state = base(self) 72 args = (self.class, base, state)

TypeError: can't pickle Variable objects ```

I would default to 3, and in the exception suggest using a different merge option. Imagine merging two datasets with different _FillValue, unit, or compression attributes.

http://docs.python.org/2/library/pickle.html#pickling-and-unpickling-normal-class-instances

Looks like this may be the only option. Based on my tests, netCDF4 is strongly antithetical to any kind of streams/piped buffers. If we go the hdf5 route, we'd have to reimplement the CDM/netcdf4 on top of hdf5, no?

I'd preserve as many of the original variables as possible, similarly to preserving _FillValue. you'll need them if you want to preserve file structure.

Fair enough.

On Sun, Feb 23, 2014 at 10:40 AM, Stephan Hoyer notifications@github.comwrote:

My preference is to stick with the presumably battle-hardened code from pandas. It would probably be slightly cleaner to use the git-python library but that adds another dependency (at least to the build) and then we'd have to debug it. I think git describe has been around for quite a while (and is unlikely to change).

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/xray/pull/15#issuecomment-35839019 .

Hmm. Would it be cleaner to use the git-python library? It would be cleaner than running a subprocess with possibly unknown versions of git. In addition, you could have it a requirement only for the build, not for installation.

Thanks for looking at that. I'll do a more thorough evaluation over the weekend!

On Fri, Feb 14, 2014 at 5:49 PM, Stephan Hoyer notifications@github.comwrote:

Thanks for @ebrevdo https://github.com/ebrevdo for taking a look! I'm pretty sure the issue here is the numpy version. I am running numpy 1.8 on my machine. It looks like we need at least numpy 1.7 for timedelta math [1], and it appears that array_equal can only compare string arrays in numpy 1.8 [2]. It's not insurmountable to work around either of these issues, but for I will increment the required version of numpy to 1.8.

[1] http://pandas.pydata.org/pandas-docs/stable/timeseries.html#numpy-1-7-compatibility [2] numpy/numpy#2686 https://github.com/numpy/numpy/issues/2686

Reply to this email directly or view it on GitHubhttps://github.com/akleeman/scidata/pull/12#issuecomment-35144276 .

Looks like a great change! I'm seeing some failing tests:

~/dev/scidata (DataView)$ python setup.py test running test running egg_info writing requirements to src/xray.egg-info/requires.txt writing src/xray.egg-info/PKG-INFO writing top-level names to src/xray.egg-info/top_level.txt writing dependency_links to src/xray.egg-info/dependency_links.txt reading manifest file 'src/xray.egg-info/SOURCES.txt' writing manifest file 'src/xray.egg-info/SOURCES.txt' running build_ext test_1d_math (test.test_array.TestArray) ... ok test_aggregate (test.test_array.TestArray) ... ERROR test_array_interface (test.test_array.TestArray) ... ok test_broadcasting_failures (test.test_array.TestArray) ... ok test_broadcasting_math (test.test_array.TestArray) ... ok test_collapse (test.test_array.TestArray) ... ok test_data (test.test_array.TestArray) ... ok test_from_stack (test.test_array.TestArray) ... ok test_groupby (test.test_array.TestArray) ... ERROR test_indexed_by (test.test_array.TestArray) ... ok test_inplace_math (test.test_array.TestArray) ... ok test_items (test.test_array.TestArray) ... ok test_properties (test.test_array.TestArray) ... ok test_repr (test.test_array.TestArray) ... ok test_transpose (test.test_array.TestArray) ... ok test_attributes (test.test_dataset.DataTest) ... SKIP: attribute checks are not yet backend specific test_coordinate (test.test_dataset.DataTest) ... ok test_copy (test.test_dataset.DataTest) ... ok test_dimension (test.test_dataset.DataTest) ... ok test_getitem (test.test_dataset.DataTest) ... ERROR test_indexed_by (test.test_dataset.DataTest) ... ok test_init (test.test_dataset.DataTest) ... ok test_iterator (test.test_dataset.DataTest) ... ok test_labeled_by (test.test_dataset.DataTest) ... ERROR test_merge (test.test_dataset.DataTest) ... ok test_rename (test.test_dataset.DataTest) ... ok test_repr (test.test_dataset.DataTest) ... ok test_select (test.test_dataset.DataTest) ... ok test_setitem (test.test_dataset.DataTest) ... ok test_to_dataframe (test.test_dataset.DataTest) ... ok test_unselect (test.test_dataset.DataTest) ... SKIP: need to write this test test_variable (test.test_dataset.DataTest) ... ok test_variable_indexing (test.test_dataset.DataTest) ... ok test_write_store (test.test_dataset.DataTest) ... ok test_attributes (test.test_dataset.NetCDF4DataTest) ... SKIP: attribute checks are not yet backend specific test_coordinate (test.test_dataset.NetCDF4DataTest) ... ok test_copy (test.test_dataset.NetCDF4DataTest) ... ok test_dimension (test.test_dataset.NetCDF4DataTest) ... ok test_dump_and_open_dataset (test.test_dataset.NetCDF4DataTest) ... ok test_getitem (test.test_dataset.NetCDF4DataTest) ... ERROR test_indexed_by (test.test_dataset.NetCDF4DataTest) ... ok test_init (test.test_dataset.NetCDF4DataTest) ... ok test_iterator (test.test_dataset.NetCDF4DataTest) ... ok test_labeled_by (test.test_dataset.NetCDF4DataTest) ... ERROR test_merge (test.test_dataset.NetCDF4DataTest) ... ok test_rename (test.test_dataset.NetCDF4DataTest) ... ok test_repr (test.test_dataset.NetCDF4DataTest) ... ok test_select (test.test_dataset.NetCDF4DataTest) ... ok test_setitem (test.test_dataset.NetCDF4DataTest) ... ok test_to_dataframe (test.test_dataset.NetCDF4DataTest) ... ok test_unselect (test.test_dataset.NetCDF4DataTest) ... SKIP: need to write this test test_variable (test.test_dataset.NetCDF4DataTest) ... ok test_variable_indexing (test.test_dataset.NetCDF4DataTest) ... ok test_write_store (test.test_dataset.NetCDF4DataTest) ... ok test_attributes (test.test_dataset.ScipyDataTest) ... SKIP: attribute checks are not yet backend specific test_coordinate (test.test_dataset.ScipyDataTest) ... ok test_copy (test.test_dataset.ScipyDataTest) ... ok test_dimension (test.test_dataset.ScipyDataTest) ... ok test_dump_and_open_dataset (test.test_dataset.ScipyDataTest) ... FAIL test_getitem (test.test_dataset.ScipyDataTest) ... ERROR test_indexed_by (test.test_dataset.ScipyDataTest) ... ok test_init (test.test_dataset.ScipyDataTest) ... ok test_iterator (test.test_dataset.ScipyDataTest) ... ok test_labeled_by (test.test_dataset.ScipyDataTest) ... ERROR test_merge (test.test_dataset.ScipyDataTest) ... ok test_rename (test.test_dataset.ScipyDataTest) ... ok test_repr (test.test_dataset.ScipyDataTest) ... ok test_select (test.test_dataset.ScipyDataTest) ... ok test_setitem (test.test_dataset.ScipyDataTest) ... ok test_to_dataframe (test.test_dataset.ScipyDataTest) ... ok test_unselect (test.test_dataset.ScipyDataTest) ... SKIP: need to write this test test_variable (test.test_dataset.ScipyDataTest) ... ok test_variable_indexing (test.test_dataset.ScipyDataTest) ... ok test_write_store (test.test_dataset.ScipyDataTest) ... ok test.test_dataset.create_test_data ... ok test_aggregate (test.test_dataset_array.TestDatasetArray) ... FAIL test_array_interface (test.test_dataset_array.TestDatasetArray) ... ok test_collapse (test.test_dataset_array.TestDatasetArray) ... ok test_dataset_getitem (test.test_dataset_array.TestDatasetArray) ... ok test_from_stack (test.test_dataset_array.TestDatasetArray) ... ok test_groupby (test.test_dataset_array.TestDatasetArray) ... FAIL test_indexed_by (test.test_dataset_array.TestDatasetArray) ... ok test_inplace_math (test.test_dataset_array.TestDatasetArray) ... ok test_intersection (test.test_dataset_array.TestDatasetArray) ... FAIL test_item_math (test.test_dataset_array.TestDatasetArray) ... ok test_items (test.test_dataset_array.TestDatasetArray) ... ok test_iteration (test.test_dataset_array.TestDatasetArray) ... ok test_labeled_by (test.test_dataset_array.TestDatasetArray) ... FAIL test_loc (test.test_dataset_array.TestDatasetArray) ... FAIL test_math (test.test_dataset_array.TestDatasetArray) ... ok test_properties (test.test_dataset_array.TestDatasetArray) ... ok test_refocus (test.test_dataset_array.TestDatasetArray) ... ok test_renamed (test.test_dataset_array.TestDatasetArray) ... ok test_frozen (test.test_utils.TestDictionaries) ... ok test_ordered_dict_intersection (test.test_utils.TestDictionaries) ... ok test_safe (test.test_utils.TestDictionaries) ... ok test_unsafe (test.test_utils.TestDictionaries) ... ok test_expanded_indexer (test.test_utils.TestIndexers) ... ok test_orthogonal_indexer (test.test_utils.TestIndexers) ... ok test (test.test_utils.TestNum2DatetimeIndex) ... ERROR

ERROR: test_aggregate (test.test_array.TestArray)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_array.py", line 240, in test_aggregate self.assertVarEqual(expected_unique, actual_unique) File "/Users/ebrevdo/dev/scidata/test/init.py", line 10, in assertVarEqual self.assertTrue(utils.variable_equal(v1, v2)) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 132, in variable_equal return np.array_equal(data1, data2) File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/numeric.py", line 1977, in array_equal return bool(logical_and.reduce(equal(a1,a2).ravel())) AttributeError: 'NotImplementedType' object has no attribute 'ravel'

ERROR: test_groupby (test.test_array.TestArray)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_array.py", line 220, in test_groupby self.assertVarEqual(expected_unique, grouped.unique_coord) File "/Users/ebrevdo/dev/scidata/test/init.py", line 10, in assertVarEqual self.assertTrue(utils.variable_equal(v1, v2)) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 132, in variable_equal return np.array_equal(data1, data2) File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/numeric.py", line 1977, in array_equal return bool(logical_and.reduce(equal(a1,a2).ravel())) AttributeError: 'NotImplementedType' object has no attribute 'ravel'

ERROR: test_getitem (test.test_dataset.DataTest)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 325, in test_getitem {'units': 'days since 2000-01-01'}) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable return self.add_variable(name, v) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable return self.set_variable(name, var) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable self.indices.build_index(name) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index self.cache[key] = self.dataset._create_index(key) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index attr.get('calendar')) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex dates = first_time_delta * num_delta + np.datetime64(first_dates[0]) TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_labeled_by (test.test_dataset.DataTest)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 234, in test_labeled_by {'units': 'days since 2000-01-01'}) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable return self.add_variable(name, v) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable return self.set_variable(name, var) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable self.indices.build_index(name) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index self.cache[key] = self.dataset._create_index(key) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index attr.get('calendar')) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex dates = first_time_delta * num_delta + np.datetime64(first_dates[0]) TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_getitem (test.test_dataset.NetCDF4DataTest)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 325, in test_getitem {'units': 'days since 2000-01-01'}) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable return self.add_variable(name, v) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable return self.set_variable(name, var) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable self.indices.build_index(name) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index self.cache[key] = self.dataset._create_index(key) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index attr.get('calendar')) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex dates = first_time_delta * num_delta + np.datetime64(first_dates[0]) TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_labeled_by (test.test_dataset.NetCDF4DataTest)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 234, in test_labeled_by {'units': 'days since 2000-01-01'}) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable return self.add_variable(name, v) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable return self.set_variable(name, var) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable self.indices.build_index(name) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index self.cache[key] = self.dataset._create_index(key) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index attr.get('calendar')) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex dates = first_time_delta * num_delta + np.datetime64(first_dates[0]) TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_getitem (test.test_dataset.ScipyDataTest)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 325, in test_getitem {'units': 'days since 2000-01-01'}) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable return self.add_variable(name, v) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable return self.set_variable(name, var) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable self.indices.build_index(name) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index self.cache[key] = self.dataset._create_index(key) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index attr.get('calendar')) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex dates = first_time_delta * num_delta + np.datetime64(first_dates[0]) TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test_labeled_by (test.test_dataset.ScipyDataTest)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 234, in test_labeled_by {'units': 'days since 2000-01-01'}) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 464, in create_variable return self.add_variable(name, v) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 534, in add_variable return self.set_variable(name, var) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 595, in set_variable self.indices.build_index(name) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 109, in build_index self.cache[key] = self.dataset._create_index(key) File "/Users/ebrevdo/dev/scidata/src/xray/dataset.py", line 224, in _create_index attr.get('calendar')) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex dates = first_time_delta * num_delta + np.datetime64(first_dates[0]) TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

ERROR: test (test.test_utils.TestNum2DatetimeIndex)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_utils.py", line 68, in test actual = utils.num2datetimeindex(num_dates, units, calendar) File "/Users/ebrevdo/dev/scidata/src/xray/utils.py", line 106, in num2datetimeindex dates = first_time_delta * num_delta + np.datetime64(first_dates[0]) TypeError: ufunc 'multiply' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule 'safe'

FAIL: test_dump_and_open_dataset (test.test_dataset.ScipyDataTest)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset.py", line 404, in test_dump_and_open_dataset self.assertEquals(expected, actual) AssertionError: <xray.Dataset (time: 1000, @dim1: 100, @dim2: 50, @dim3: 10): var1 var2 var3> != <xray.Dataset (@dim2: 50, @dim3: 10, @dim1: 100, time: 1000): var1 var3 var2>

FAIL: test_aggregate (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 213, in test_aggregate self.assertViewEqual(expected, actual) File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual self.assertEqual(dv1.dataset, dv2.dataset) AssertionError: <xray.Dataset (@x: 10, @abc: 3): foo> != <xray.Dataset (@x: 10, @abc: 3): foo>

FAIL: test_groupby (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 189, in test_groupby grouped.collapse(np.sum, dimension=None)) File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual self.assertEqual(dv1.dataset, dv2.dataset) AssertionError: <xray.Dataset (@abc: 3): foo> != <xray.Dataset (@abc: 3): foo>

FAIL: test_intersection (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 240, in test_intersection self.assertViewEqual(dv1, self.dv[:5]) File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual self.assertEqual(dv1.dataset, dv2.dataset) AssertionError: <xray.Dataset (@x: 5, @y: 20): foo> != <xray.Dataset (@x: 5, @y: 20): foo>

FAIL: test_labeled_by (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 75, in test_labeled_by self.assertViewEqual(self.dv, self.dv.labeled_by(x=slice(None))) File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual self.assertEqual(dv1.dataset, dv2.dataset) AssertionError: <xray.Dataset (@x: 10, @y: 20): foo> != <xray.Dataset (@x: 10, @y: 20): foo>

FAIL: test_loc (test.test_dataset_array.TestDatasetArray)

Traceback (most recent call last): File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 81, in test_loc self.assertViewEqual(self.dv[:3], self.dv.loc[:'c']) File "/Users/ebrevdo/dev/scidata/test/test_dataset_array.py", line 9, in assertViewEqual self.assertEqual(dv1.dataset, dv2.dataset) AssertionError: <xray.Dataset (@x: 3, @y: 20): foo> != <xray.Dataset (@x: 3, @y: 20): foo>

Ran 100 tests in 1.248s

FAILED (failures=6, errors=9, skipped=6)

Nvm. Will review over the next day or two.

In that case, planning to close this one?