issue_comments

Is this something that we want to mandate that backends provide?

I think it would be better to do this systematically, e.g., inside xarray.open_dataset(). We would need to verify that filename_or_obj is provided as a string, but if so we could add it into encoding on the Dataset object.

Yes, that sounds great! Potentially this would be a good opportunity for a doc update, too.

On Mon, Nov 19, 2018 at 6:36 AM Julia Signell notifications@github.com wrote:

Should I add a test that expects .encoding['source'] to ensure its continued presence?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/2550#issuecomment-439913493, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1tyOrzPg3dUX2cPolk51Uoa4i7j5ks5uwsH2gaJpZM4YVZ1J .

I'm not sure .encoding['source'] should really be relied upon -- it wasn't really an intentional API decision. But I guess it's harmless enough to include it...

The danger with inspecting user provided functions is that it's pretty fragile, e.g., it fails if you use provide a signature like args, *kwargs (which can happen pretty easily with decorators). Probably the best option is to come up with a new keyword argument to replace "preprocess" and to deprecate the current preprocess (if we can think of another good name). We could also do a deprecation cycle with FutureWarning, but that's pretty painful.

On Fri, Nov 9, 2018 at 12:29 PM Julia Signell notifications@github.com wrote:

Maybe we can inspect the preprocess function like this:

preprocess = lambda a, b: print(a, b)>>> preprocess .code.co_varnames ('a', 'b')

This response is ordered, so the first one can always be ds regardless of its name and then we can look for special names (like filename) in the rest.

From this answer: https://stackoverflow.com/a/4051447/4021797

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/2550#issuecomment-437433736, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1kvSMx83hACWxXD3tOyIHGqyqum8ks5utbtigaJpZM4YVZ1J .

@jhamman The problem is that xarray needs way to figure out what arguments it can safely pass to preprocess, i.e., it needs to inspect the proprocess function and see if it can handle a filename argument. It's not obvious what the best way to do this in a backwards compatible way is...

Hmm. It really seems like the preprocess function should pass in the file-name along with the dataset.