issue_comments

Given that this error only is caused when Python is shutting down, which is exactly a case in which we do not need to clean up open file objects, maybe we can remove the __del__ instead?

Something like: ```python import atexit

@atexit.register def _remove_del_method(): # We don't need to close unclosed files at program exit, # and may not be able to do, because Python is cleaning up # imports. del CachingFileManager.del ```

(I have not tested this!)

that appears to work on both my laptop and my local HPC, and is arguably a lot easier to implement / understand as we don't need to make sure all the globals we use are still available (which in this case would be acquire, OPTIONS, warnings, and RuntimeWarning).

Edit: let me change the PR to do that instead

Have you verfied that this fixes things at least on your machine?

I thought I did, but apparently something changed: right now it fails because OPTIONS is not available anymore, so we might have to add a reference to that, as well. Additionally, for the warning we use warnings.warn and RuntimeWarning that might also have disappeared already (but those are standard library / builtins, so hopefully not?)

In any case, you can verify this, too: - create a new environment using mamba create -n test python=3.11 numpy pandas packaging pooch netcdf4 and activate it - run pip install 'dask[array]' to install dask without distributed (this appears to make a difference for me, not sure if that's the same elsewhere) - editable-install xarray so we can easily switch between branches - run python -c 'import xarray as xr; ds = xr.tutorial.open_dataset("air_temperature", chunks={})'

This should print an error for main and shouldn't with this branch (confirmed on my local HPC).

the issue is that this doesn't occur on normal garbage collection but only on interpreter shutdown. So really, I don't think we have any way to test this using pytest as that itself is written in python (unless of course we can make use of sub-interpreters, but that might be more trouble than it's worth).

Maybe you can add a test that creates a cachingFilemanager object, then deletes it, then run gc.collect() and check somehow if it works? But no idea how pytest interferes with this or how you can ensure that there are no references to the module anymore?

I agree that this seems very hard to test!

Have you verfied that this fixes things at least on your machine?

I think this is intended (though certainly not very easy to get right): see the second part of the warning in the __del__ documentation.

You are right, that warning is exactly what is causing the issues.

I would have thought that the global (or module level here) variable/function aquire should have at least one reference until after the deletion of the object.

I think this is intended (though certainly not very easy to get right): see the second part of the warning in the __del__ documentation.

That's quite a weird bug. I would have thought that the global (or module level here) variable/function aquire should have at least one reference until after the deletion of the object. Is that a bug in pythons garbage collection?

Or does the garbage collection already start when calling del and does not wait for the completion of the __del__ method?