html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/issues/992#issuecomment-269510467,https://api.github.com/repos/pydata/xarray/issues/992,269510467,MDEyOklzc3VlQ29tbWVudDI2OTUxMDQ2Nw==,2443309,2016-12-28T17:29:57Z,2016-12-28T17:29:57Z,MEMBER,"For those of you who are interested in this feature, I'd appreciate your feedback on #1170.","{""total_count"": 1, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 1, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-267476189,https://api.github.com/repos/pydata/xarray/issues/992,267476189,MDEyOklzc3VlQ29tbWVudDI2NzQ3NjE4OQ==,6864797,2016-12-15T23:34:34Z,2016-12-15T23:34:34Z,NONE,"@jhamman : Great!
@chrisb13 : I've been using the StackOverflow hack and it does work.","{""total_count"": 2, ""+1"": 2, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-267185985,https://api.github.com/repos/pydata/xarray/issues/992,267185985,MDEyOklzc3VlQ29tbWVudDI2NzE4NTk4NQ==,2443309,2016-12-14T23:09:50Z,2016-12-14T23:09:50Z,MEMBER,All - I'm actively working on this. I have an initial implementation and will get it cleaned up here in the next few days. Stay tuned.,"{""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-267177054,https://api.github.com/repos/pydata/xarray/issues/992,267177054,MDEyOklzc3VlQ29tbWVudDI2NzE3NzA1NA==,5499680,2016-12-14T22:28:31Z,2016-12-14T22:28:31Z,NONE,"Useful for me too.
For the time being there's also this hack (I haven't tested)..
http://stackoverflow.com/questions/28598485/how-to-convert-fixed-size-dimension-to-unlimited-in-a-netcdf-file","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-266245983,https://api.github.com/repos/pydata/xarray/issues/992,266245983,MDEyOklzc3VlQ29tbWVudDI2NjI0NTk4Mw==,6864797,2016-12-10T23:01:15Z,2016-12-10T23:01:15Z,NONE,The ability to set unlimited dimension would be really useful for me as well.,"{""total_count"": 4, ""+1"": 4, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-258949295,https://api.github.com/repos/pydata/xarray/issues/992,258949295,MDEyOklzc3VlQ29tbWVudDI1ODk0OTI5NQ==,3404817,2016-11-07T20:14:37Z,2016-11-07T20:14:37Z,CONTRIBUTOR,"Great to see you are pushing forward on this issue @jhamman and @shoyer. I would really have liked to contribute here, but it seems like there are quite some design choices to make, which are better left in your hands.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-258877559,https://api.github.com/repos/pydata/xarray/issues/992,258877559,MDEyOklzc3VlQ29tbWVudDI1ODg3NzU1OQ==,1217238,2016-11-07T16:05:04Z,2016-11-07T16:05:04Z,MEMBER,"Agreed, it's awkward to have this information on variables.
I was somewhat opposed to adding more state to the `Dataset` object but it seems like the necessary solution here. I'm not sure we need it in the `Dataset` constructor though -- could just have `encoding` as an attribute you need to modify. Honestly, could probably do the same for `DataArray.encoding` -- it's pretty low level.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-258834200,https://api.github.com/repos/pydata/xarray/issues/992,258834200,MDEyOklzc3VlQ29tbWVudDI1ODgzNDIwMA==,2443309,2016-11-07T13:24:20Z,2016-11-07T13:24:20Z,MEMBER,"@shoyer -
I played around with this a bit yesterday. I have two implementation questions:
- for a dataset with multiple variables, putting the dimension encoding on the `Variable` easily leads to conflicts. For example, one variable's encoding may say that dimension `x` is unlimited while another doesn't. How should we handle these conflicts.
- are we apposed to a encoding attribute on the dataset object?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243546130,https://api.github.com/repos/pydata/xarray/issues/992,243546130,MDEyOklzc3VlQ29tbWVudDI0MzU0NjEzMA==,1217238,2016-08-30T19:07:19Z,2016-08-30T19:07:19Z,MEMBER,"> However, when the dataset is indexed/subset/resampled along the unlimited dimension, it would make sense that its state is dropped. But that would require a lot of ifs and buts, so I suggest we leave that aside for now.
This is exactly how `Variable.encoding` currently works: any operation that creates a new variable from the original variable drops the encoding.
If we put this encoding information on the variable corresponding to the dimension, any time you save a Dataset using that exact same dimension variable, it would be saved as unlimited size. So if you only modify other dimensions (e.g., with resampling or indexing), the unlimited dimension would indeed persist, as you desire.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243367823,https://api.github.com/repos/pydata/xarray/issues/992,243367823,MDEyOklzc3VlQ29tbWVudDI0MzM2NzgyMw==,3404817,2016-08-30T08:22:32Z,2016-08-30T08:22:32Z,CONTRIBUTOR,"@jhamman sorry, I only now saw that you pointed to a previous issue on the same topic (with exactly the same considerations). I did not find that issue when I searched (for ""unlimited"").
You were against changing `to_netcdf` [then](https://github.com/pydata/xarray/issues/678#issuecomment-164082151). Are you still?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243365966,https://api.github.com/repos/pydata/xarray/issues/992,243365966,MDEyOklzc3VlQ29tbWVudDI0MzM2NTk2Ng==,3404817,2016-08-30T08:14:08Z,2016-08-30T08:14:08Z,CONTRIBUTOR,"The above solution would not require much more than that,
in [`set_necessary_dimensions`](https://github.com/pydata/xarray/blob/master/xarray/backends/common.py#L222-L225),
```
def set_necessary_dimensions(self, variable):
for d, l in zip(variable.dims, variable.shape):
if d not in self.dimensions:
self.set_dimension(d, l)
```
would become
```
def set_necessary_dimensions(self, variable):
for d, l in zip(variable.dims, variable.shape):
if d in self._unlimited_dimensions:
l = None
if d not in self.dimensions:
self.set_dimension(d, l)
```
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243364284,https://api.github.com/repos/pydata/xarray/issues/992,243364284,MDEyOklzc3VlQ29tbWVudDI0MzM2NDI4NA==,3404817,2016-08-30T08:06:49Z,2016-08-30T08:07:07Z,CONTRIBUTOR,"But maybe the `encoding` dict is not the way to go after all, since it contains entries per **variable**, while it is the **dimension** that must be unlimited.
Currently the dataset variables can be created in any order and their necessary dimensions created whenever needed (in the [`set_necessary_dimensions`](https://github.com/pydata/xarray/blob/master/xarray/backends/common.py#L222-L225) function). I would **not** like to change that logic (e.g. towards creating all dimensions required by all variables first, before adding the data variables).
So how about a new kw argument to `to_netcdf`, like
```
ds.to_netcdf(unlimited_dimensions=['time'])
```
or
```
ds.to_netcdf(dimension_unlimited={'time': True})
```
(the second option better for explicitly setting {'time': False})?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243350429,https://api.github.com/repos/pydata/xarray/issues/992,243350429,MDEyOklzc3VlQ29tbWVudDI0MzM1MDQyOQ==,3404817,2016-08-30T06:59:14Z,2016-08-30T06:59:14Z,CONTRIBUTOR,"I think it makes sense to preserve the `UNLIMITED` state through read/write. In my case, I subset a netCDF file along `lat` and `lon` dimensions, leaving the `time` dimension untouched and would therefore expect it to pass unchanged through `xarray` IO (staying `UNLIMITED`).
However, when the dataset is indexed/subset/resampled along the unlimited dimension, it would make sense that its state is dropped. But that would require a lot of ifs and buts, so I suggest we leave that aside for now.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243287795,https://api.github.com/repos/pydata/xarray/issues/992,243287795,MDEyOklzc3VlQ29tbWVudDI0MzI4Nzc5NQ==,1217238,2016-08-29T23:24:21Z,2016-08-29T23:24:21Z,MEMBER,"Yes, we could put this in encoding if we want to preserve through
reading/writing files. NetCDF4 supports multiple unlimited dimensions.
Netcdf3 does not.
On Mon, Aug 29, 2016 at 1:56 PM Jonas notifications@github.com wrote:
> OK, I'd be up for taking a shot at it.
>
> Since it is per-variable and specific to netCDF, I guess the perfect place
> to add this is in the encoding dictionary that you can pass to to_netcdf
> https://github.com/pydata/xarray/blob/606e1d9c7efd72e10b530a688d6ef870e8ec1843/xarray/backends/api.py#L316,
> right? Maybe as key unlimited? E.g.
>
> ds.to_netcdf(encoding={'time': dict(unlimited=True)})
>
> I need to look up whether netCDF allows for defining more than one
> unlimited dimension, otherwise that must throw an error.
>
> And then it is just about passing None as length to CreateDimension, at
> least in netCDF4 and scipy.io.netcdf. But I did not look into how xarray
> handles that under the hood.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> https://github.com/pydata/xarray/issues/992#issuecomment-243253510, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/ABKS1mTPcRqf3JI-yRk4wnIQ_xgKE3Wvks5qk0eXgaJpZM4JveCA
> .
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243253510,https://api.github.com/repos/pydata/xarray/issues/992,243253510,MDEyOklzc3VlQ29tbWVudDI0MzI1MzUxMA==,3404817,2016-08-29T20:56:55Z,2016-08-29T20:56:55Z,CONTRIBUTOR,"OK, I'd be up for taking a shot at it.
Since it is per-variable and specific to netCDF, I guess the perfect place to add this is in the `encoding` dictionary that you can pass to [`to_netcdf`](https://github.com/pydata/xarray/blob/606e1d9c7efd72e10b530a688d6ef870e8ec1843/xarray/backends/api.py#L316), right? Maybe as key `unlimited`? E.g.
```
ds.to_netcdf(encoding={'time': dict(unlimited=True)})
```
I need to look up whether netCDF allows for defining more than one unlimited dimension, otherwise that must throw an error.
And then it is just about passing `None` as length to `CreateDimension`, at least in `netCDF4` and `scipy.io.netcdf`. But I did not look into how xarray handles that under the hood.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243177100,https://api.github.com/repos/pydata/xarray/issues/992,243177100,MDEyOklzc3VlQ29tbWVudDI0MzE3NzEwMA==,2443309,2016-08-29T16:31:45Z,2016-08-29T16:31:45Z,MEMBER,"xref: https://github.com/pydata/xarray/issues/678
I'm also a +1 on this.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243168308,https://api.github.com/repos/pydata/xarray/issues/992,243168308,MDEyOklzc3VlQ29tbWVudDI0MzE2ODMwOA==,1197350,2016-08-29T16:00:08Z,2016-08-29T16:00:08Z,MEMBER,"👍 from me. In the broader netCDF ecosystem, it can be pretty important to have record dimensions for various reasons, appending being the main one.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358
https://github.com/pydata/xarray/issues/992#issuecomment-243167445,https://api.github.com/repos/pydata/xarray/issues/992,243167445,MDEyOklzc3VlQ29tbWVudDI0MzE2NzQ0NQ==,1217238,2016-08-29T15:57:27Z,2016-08-29T15:57:27Z,MEMBER,"Currently it's not supported, but yes, we could absolutely add it as an option. I would be happy to add this functionality if someone makes a pull request. This won't be very useful for editing the file with xarray of course because we don't support editing netcdf files without making a complete copy.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,173773358