issue_comments

In a future where encoding has been removed from Xarray's data model entirely, would open_dataset_with_encoding, or whatever name gets settled on, still exist? It's not clear to me if removal from the data model means just removing it from Xarray's data structures, or if it also means removing it from Xarray's APIs.

In the hypothetical invocation open_dataset(..., return_encoding=True), do you envision the returned encoding as being a separate returned object, or would it still be an attribute on the Dataset object?

My expectation was that this would be a separate object, e.g., dataset, encoding = xarray.open_dataset(..., return_encoding=True), where encoding is a dict providing the encoding on each variable, and which could be passed as the encoding argument into to_netcdf(). That said, I can see how keeping encoding as variable attributes could also be convenient.

"disable all encoding propagation by discarding encoding attributes once a Dataset has been modified" would be an intermediate step, on the route to removing encoding from Xarray's data model entirely entirely.

(As a side note, I would probably spell this as open_dataset_with_encoding rather than having a function with a variable return signature.)

In the hypothetical invocation open_dataset(..., return_encoding=True), do you envision the returned encoding as being a separate returned object, or would it still be an attribute on the Dataset object? I'm guessing the latter, because the subsequent statement 'disable all encoding propagation by discarding encoding attributes once a Dataset has been modified' doesn't make much sense to me for the former. If so, after encoding attributes are discarded, would there still be an encoding attribute on the Dataset object that the user could reset to the values prior to the Dataset modification? This would enable the user to propagate encoding values through their workflow.

We should also consider a configuration option to automatically drop encoding.

This issue was discussed at this week's dev meeting. I will summarize what we discussed:

1. General agreement that propagating encoding through arbitrary operations (e.g. slice, chunk, computation) leads to inconsistent states that are hard to protect against. This often leads to problems when serializing datasets in our backends.
2. The primary benefit of keeping encoding on Xarray objects is the ability to exactly roundtrip datasets. However, this benefit is less obvious after a dataset has been modified.
3. We currently have two APIs for setting encoding (e.g. to_netcdf(..., encoding={...}) and ds.encoding = {...}). We should change this by deprecating setting encoding on Xarray objects using the .encoding property.
4. We can move towards providing utilities that expose a dataset's source encoding (e.g. open_dataset(..., return_encoding=True).

Specific action items that can happen now: - [x] add reset_encoding to Dataset/DatAarray api (https://github.com/pydata/xarray/issues/7686) - [ ] add a DeprecationWarning to the @property.setter for encoding on Dataset/DatAarray/Variable - [ ] document the change in callout in the Xarray user guide.

Longer term action items: - [ ] add option to backend readers to keep / discard interpreted encoding attributes - [ ] disable all encoding propagation by discarding encoding attributes once a Dataset has been modified.

See also https://github.com/pydata/xarray/issues/7686. The ideas presented here are also great!