html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue
https://github.com/pydata/xarray/pull/2706#issuecomment-506383042,https://api.github.com/repos/pydata/xarray/issues/2706,506383042,MDEyOklzc3VlQ29tbWVudDUwNjM4MzA0Mg==,1197350,2019-06-27T14:59:52Z,2019-06-27T14:59:52Z,MEMBER,"> adding a new variable currently errors if we don't provide the `append_dim` argument:
Is this scenario now covered by the tests? Sorry if the answer is obvious; it's hard for me to discern just by looking at the code.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-505903058,https://api.github.com/repos/pydata/xarray/issues/2706,505903058,MDEyOklzc3VlQ29tbWVudDUwNTkwMzA1OA==,1197350,2019-06-26T14:34:27Z,2019-06-26T14:34:27Z,MEMBER,Thanks @shoyer for your more careful review of this PR. As usual you pick up on all the important details.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-502700081,https://api.github.com/repos/pydata/xarray/issues/2706,502700081,MDEyOklzc3VlQ29tbWVudDUwMjcwMDA4MQ==,1197350,2019-06-17T14:13:45Z,2019-06-17T14:13:45Z,MEMBER,@shikharsg - are the issues you found in https://github.com/pydata/xarray/pull/2706#issuecomment-498194520 now resolved and covered by tests?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-502699808,https://api.github.com/repos/pydata/xarray/issues/2706,502699808,MDEyOklzc3VlQ29tbWVudDUwMjY5OTgwOA==,1197350,2019-06-17T14:12:59Z,2019-06-17T14:12:59Z,MEMBER,"@jendrikjoe - thanks for digging in and finding this important issue!
This PR has been hanging around for a long time. (A lot of that is on me!) It would be good to get something merged soon. Here's what I propose.
- Identify which datatypes can easily be appended now (e.g. floats, etc.) and which cannot (variable length strings)
- Raise an error if append is called on the incompatible datatypes
- Move forward with this PR, which is otherwise very nearly ready
- Open a new issue to keep track of the outstanding incompatible types, which require upstream resolution in zarr
How does that sound to everyone?
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-498227133,https://api.github.com/repos/pydata/xarray/issues/2706,498227133,MDEyOklzc3VlQ29tbWVudDQ5ODIyNzEzMw==,1197350,2019-06-03T11:58:19Z,2019-06-03T11:58:19Z,MEMBER,"Let’s make sure this new scenario is covered by tests!
Sent from my iPhone
> On Jun 3, 2019, at 6:40 AM, Jendrik Jördening wrote:
>
> Gave you the permissions @shikharsg
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-494062757,https://api.github.com/repos/pydata/xarray/issues/2706,494062757,MDEyOklzc3VlQ29tbWVudDQ5NDA2Mjc1Nw==,1197350,2019-05-20T16:38:24Z,2019-05-20T16:38:24Z,MEMBER,"Hi @davidbrochart. I'm really sorry it takes me so long between reviews of your PR. It is very important work, and I appreciate your continued patience.
I looked at your new code, and I noticed that `chunk_dim` does not appear in the tests. I think it is important to test this parameter and verify that it works as expected. (This would also help me understand how it works, since it's not totally clear from the docs.)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-484542216,https://api.github.com/repos/pydata/xarray/issues/2706,484542216,MDEyOklzc3VlQ29tbWVudDQ4NDU0MjIxNg==,1197350,2019-04-18T14:49:54Z,2019-04-18T14:49:54Z,MEMBER,Where do we stand on this PR? @davidbrochart - do you feel this is ready for a final review? Or do you want advice or feedback on anything?,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-478374296,https://api.github.com/repos/pydata/xarray/issues/2706,478374296,MDEyOklzc3VlQ29tbWVudDQ3ODM3NDI5Ng==,1197350,2019-03-31T19:47:49Z,2019-03-31T19:47:49Z,MEMBER,@davidbrochart I would personally be happy to see anyone work on this. I'm sure @jendrikjoe would not mind if we make it a team effort!,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-459813678,https://api.github.com/repos/pydata/xarray/issues/2706,459813678,MDEyOklzc3VlQ29tbWVudDQ1OTgxMzY3OA==,1197350,2019-02-01T18:07:26Z,2019-02-01T18:07:26Z,MEMBER,"> We should definitely always make sure that we write data consistently (e.g., for dates), but checking for alignment of all coordinates could be expensive/slow.
This implies we should be checking for attributes compatibility before calling `zarr.append`. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-459177873,https://api.github.com/repos/pydata/xarray/issues/2706,459177873,MDEyOklzc3VlQ29tbWVudDQ1OTE3Nzg3Mw==,1197350,2019-01-31T01:24:11Z,2019-01-31T01:24:11Z,MEMBER,"So the problem in @davidbrochart's example is that there are different encodings on the time variables in the two datasets.
When writing datetimes, xarray automatically picks an encoding (i.e. `days since 2000-01-01 00:00:00`) based on some heuristics. When serializing the dataset, this encoding is used to encode the `datetime64[ns]` dtype into a different dtype, and the encoding is placed in the attributes of the store. When you open the dataset, the encoding is automatically decoded according to CF conventions. This can be disabled by using `decode_cf=False` or `decode_times=False` when you open the dataset.
In this case, xarray's heuristics are picking different encodings for the two dates. You could make this example work by manually specifying encoding on the appended dataset to be the same as the original.
This example illustrates the need for some sort of compatibility checks between the target dataset and the appended dataset. For example, checking for attribute compatibility would have caught this error.
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-458692011,https://api.github.com/repos/pydata/xarray/issues/2706,458692011,MDEyOklzc3VlQ29tbWVudDQ1ODY5MjAxMQ==,1197350,2019-01-29T20:19:58Z,2019-01-29T20:19:58Z,MEMBER,"Ok, with the example, I can see a bit better how this works.
Here is my main concern: there doesn't appear to be any alignment checking between the target dataset and the new data. The only check that happens is whether a variable with the same name already exists in the target store, if so, append is used (rather than creating a new array). What if the coordinates differ? What if the attributes differ?
I'm not sure this is a deal-breaker. But we should be very clear about this in the docs.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148
https://github.com/pydata/xarray/pull/2706#issuecomment-457741867,https://api.github.com/repos/pydata/xarray/issues/2706,457741867,MDEyOklzc3VlQ29tbWVudDQ1Nzc0MTg2Nw==,1197350,2019-01-25T21:49:26Z,2019-01-25T21:49:26Z,MEMBER,"Hi @jendrikjoe -- thanks for submitting a PR to address one of the most important issues in xarray (IMHO)! I am very excited about your contribution and am looking forward to getting this feature merged.
I have many questions about how this works. I think the best way to move forward is to wait until we have a test for the append feature which involves the following steps:
- Write a dataset to a zarr store
- Open the store in append mode
- Append data along a particular dimension
Seeing the code that accomplishes this will help clarify for me what is happening.
Thanks again for your contribution, and welcome to xarray!","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,402908148