html_url,issue_url,id,node_id,user,created_at,updated_at,author_association,body,reactions,performed_via_github_app,issue https://github.com/pydata/xarray/issues/630#issuecomment-459457428,https://api.github.com/repos/pydata/xarray/issues/630,459457428,MDEyOklzc3VlQ29tbWVudDQ1OTQ1NzQyOA==,1217238,2019-01-31T18:40:57Z,2019-01-31T18:40:57Z,MEMBER,"The original example how gives an error about `dim_1` have different size, but you can still reproduce the behavior if you provide explicit labels for `dim_1`, e.g., ``` ds = Dataset({""a"": DataArray(np.zeros((3, 4)))}, {'dim_1': range(4)}) ds[""b""] = b = DataArray(np.zeros((3, 3)), {'dim_1': range(3)}) # !!! we implicitly fill the last column with nans. b[0, 0] = 1 print(ds[""b""][0, 0]) # ==> prints 0 ``` The `copy` argument is still a good idea. We could do: - `copy=True` always copy - `copy=None` copy if necessary (default) - `copy=False` never copy (raise if not possible) (Note that `copy=False` in align currently means ""copy if necessary"", so this would be a breaking change if users are setting that explicitly)","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-458897417,https://api.github.com/repos/pydata/xarray/issues/630,458897417,MDEyOklzc3VlQ29tbWVudDQ1ODg5NzQxNw==,1322974,2019-01-30T10:42:17Z,2019-01-30T10:42:17Z,CONTRIBUTOR,"Looks like the implicit nan-filling behavior is not present anymore, which likely killed the issue?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-458894587,https://api.github.com/repos/pydata/xarray/issues/630,458894587,MDEyOklzc3VlQ29tbWVudDQ1ODg5NDU4Nw==,26384082,2019-01-30T10:33:24Z,2019-01-30T10:33:24Z,NONE,"In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity If this issue remains relevant, please comment here; otherwise it will be marked as closed automatically ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-152290305,https://api.github.com/repos/pydata/xarray/issues/630,152290305,MDEyOklzc3VlQ29tbWVudDE1MjI5MDMwNQ==,1217238,2015-10-29T19:09:57Z,2015-10-29T19:09:57Z,MEMBER,"Yes, that's a great idea! We could add a copy=False argument to both the Dataset and DataArray constructors, like the align function. On Thu, Oct 29, 2015 at 11:58 AM, Joe Hamman notifications@github.com wrote: > ## Okay, so the `Dataset` constructor needs to create a copy of the metadata. Could we include a argument to the constructor that copies the `.values` too? Obviously there is a performance hit there but it would allow users to be explicit about their own memory handling. Just an idea. > > Reply to this email directly or view it on GitHub: > https://github.com/xray/xray/issues/630#issuecomment-152287198 ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-152287198,https://api.github.com/repos/pydata/xarray/issues/630,152287198,MDEyOklzc3VlQ29tbWVudDE1MjI4NzE5OA==,2443309,2015-10-29T18:58:43Z,2015-10-29T18:58:43Z,MEMBER,"Okay, so the `Dataset` constructor needs to create a copy of the metadata. Could we include a argument to the constructor that copies the `.values` too? Obviously there is a performance hit there but it would allow users to be explicit about their own memory handling. Just an idea. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-152270844,https://api.github.com/repos/pydata/xarray/issues/630,152270844,MDEyOklzc3VlQ29tbWVudDE1MjI3MDg0NA==,1217238,2015-10-29T18:14:20Z,2015-10-29T18:14:20Z,MEMBER,"> I think the most confusing part is that this the modifications will trickle down sometimes -- even though I'd prefer them to be always propagated, I would still mind much less if they were never propagated. I agree! I am going to try to clarify and clean this up a little bit for the next version of xray. The only part that should ever trickle back is the `values` (because that's unavoidable for high performance). Right now, modifications to the metadata (`.attrs`) can also sometimes trickle back. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-152269091,https://api.github.com/repos/pydata/xarray/issues/630,152269091,MDEyOklzc3VlQ29tbWVudDE1MjI2OTA5MQ==,1322974,2015-10-29T18:07:54Z,2015-10-29T18:07:54Z,CONTRIBUTOR,"> If you put arrays in a Dataset, you should not count on modifications to the arrays in the Dataset trickling back to the original arrays. I think the most confusing part is that this the modifications will trickle down _sometimes_ -- even though I'd prefer them to be always propagated, I would still mind much less if they were never propagated. At the end of the day it's not my call, feel free to close the issue if you want. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-152265953,https://api.github.com/repos/pydata/xarray/issues/630,152265953,MDEyOklzc3VlQ29tbWVudDE1MjI2NTk1Mw==,1217238,2015-10-29T17:56:32Z,2015-10-29T17:56:32Z,MEMBER,"> If it is the latter, one can just modify d.values and e.values (as well as their coordinates systems) to point to the new ndarrays, right? Yes, in principle we could do that. But it's very surprising for a function to modify it's input arguments, so usually we try to avoid things like that. If you put arrays in a Dataset, you should not count on modifications to the arrays in the Dataset trickling back to the original arrays. Generally, I would recommend something like the following: ``` d = DataArray(...) e = DataArray(...) d, e = xray.align(d, e, join='outer') <work to initialize d and e> ds = Dataset({""d"": d, ""e"": e}) ``` or ``` d = DataArray(...) e = DataArray(...) ds = Dataset({""d"": d, ""e"": e}) <work to initialize ds.d and ds.e> ``` ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-152105174,https://api.github.com/repos/pydata/xarray/issues/630,152105174,MDEyOklzc3VlQ29tbWVudDE1MjEwNTE3NA==,1322974,2015-10-29T07:49:23Z,2015-10-29T07:49:23Z,CONTRIBUTOR,"Sorry to raise the issue again but I just noticed that I had a couple of other places where I wrote quite naturally ``` d = DataArray(...) e = DataArray(...) ds = Dataset({""d"": d, ""e"": e}) <work to initialize d and e> ``` which is equivalent to the form I was mentioning earlier (whether `ds` will actually get initialized properly depends on whether `d` and `e` have exactly matching coordinates). To be honest I don't really understand what you mean by ""Unfortunately, it's impossible to make such automatically aligned arrays views in general, because of numpy's memory model."": when the line `ds = Dataset(...)` gets executed, the ndarrays `ds.d.values` and `ds.e.values` either point to the old ndarrays `d.values` and `e.values` or to some newly created, coordinate-aligned ndarrays. If it is the latter, one can just modify `d.values` and `e.values` (as well as their coordinates systems) to point to the new ndarrays, right? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-150732823,https://api.github.com/repos/pydata/xarray/issues/630,150732823,MDEyOklzc3VlQ29tbWVudDE1MDczMjgyMw==,1217238,2015-10-24T01:29:26Z,2015-10-24T01:29:26Z,MEMBER,"@anntzer Yes, that would be a reasonable thing to state in the docs. Though I'm not sure how widespread that pattern is -- this is the first time I've seen it used. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-150044109,https://api.github.com/repos/pydata/xarray/issues/630,150044109,MDEyOklzc3VlQ29tbWVudDE1MDA0NDEwOQ==,1322974,2015-10-21T22:37:56Z,2015-10-21T22:37:56Z,CONTRIBUTOR,"Thinking about it again, I think there could just be a mention in the docs that the pattern ``` foo = dataset[key] = DataArray(...) ``` should be avoided and written as ``` dataset[key] = DataArray(...) foo = dataset[key] ``` That should be good enough? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-150007242,https://api.github.com/repos/pydata/xarray/issues/630,150007242,MDEyOklzc3VlQ29tbWVudDE1MDAwNzI0Mg==,1217238,2015-10-21T19:53:42Z,2015-10-21T19:53:42Z,MEMBER,"> ""If you use the DataArray constructor instead of inserting an item in a Dataset, the array values will always be a view."" > Can you explain what you mean here? I am using the DataArray constructor and then inserting the item in the Dataset. Yes, I should clarify -- the DataArray you create with the constructor will be a view. When you insert it into a Dataset, it may no longer be a view (if the index coordinates are not aligned). ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-149987937,https://api.github.com/repos/pydata/xarray/issues/630,149987937,MDEyOklzc3VlQ29tbWVudDE0OTk4NzkzNw==,1217238,2015-10-21T18:36:00Z,2015-10-21T19:51:46Z,MEMBER,"If you use the DataArray constructor instead of inserting an item in a Dataset, the array values will always be a view. If you pre-align the array you are about to insert using `xray.align`, it is also guaranteed to be a view. Unfortunately, it's impossible to make such automatically aligned arrays views in general, because of numpy's memory model. One thing we could do is add an option to make inserted items always a _copy_. I would probably put this on the `Dataset.merge` method, which is what `Dataset.__getitem__` uses under the hood. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-149990310,https://api.github.com/repos/pydata/xarray/issues/630,149990310,MDEyOklzc3VlQ29tbWVudDE0OTk5MDMxMA==,1322974,2015-10-21T18:45:57Z,2015-10-21T18:45:57Z,CONTRIBUTOR,"""If you use the DataArray constructor instead of inserting an item in a Dataset, the array values will always be a view."" Can you explain what you mean here? I am using the DataArray constructor and then inserting the item in the Dataset. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-149590758,https://api.github.com/repos/pydata/xarray/issues/630,149590758,MDEyOklzc3VlQ29tbWVudDE0OTU5MDc1OA==,1322974,2015-10-20T14:49:55Z,2015-10-20T14:50:09Z,CONTRIBUTOR,"I was more expecting that if the coordinates are _not_ already aligned, then the DataArray gets modified to point to the ""larger"" array (though of course there may be issues with other variables pointing to the same array, etc.). I like the pattern ``` some_explicit_dict_name[some_explicit_entry_name] = tmpvar = np.array(...) <... work work work on initializing tmpvar, expecting to initialize some_explicit_dict_name[some_explicit_entry_name]> ``` (because the explicit name is a bit too long to work with and I make it immediately clear what `tmpvar` is used for), but xray's behavior makes this work (on DataArrays) only sometimes, with no warning that this could fail to update the array I care about. Not sure what the best approach is. Perhaps ``` tmpvar = dataset.new_dataarray(*args_and_**kwargs_to_DataArray.__init__) ``` which would make sure that no one else is pointing to the internal array at that point? ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425 https://github.com/pydata/xarray/issues/630#issuecomment-149385943,https://api.github.com/repos/pydata/xarray/issues/630,149385943,MDEyOklzc3VlQ29tbWVudDE0OTM4NTk0Mw==,1217238,2015-10-20T00:38:19Z,2015-10-20T00:38:19Z,MEMBER,"Yes, this is certainly the case -- do you suggest doing this differently? If coordinates are already aligned, I would rather not do unnecessary copies -- this could be quite expensive. If this was surprising to you, then it is certainly worth documenting somewhere. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",,112253425