issue_comments: 399540641

Again, I think the first big challenge here is writing fast approximate union/intersection algorithms. Then we can figure out how to wire them into the pandas/xarray API :).

On Fri, Jun 22, 2018 at 10:42 AM Benjamin Root notifications@github.com wrote:

Ok, I see how you implemented it for pandas's reindex. You essentially inserted an inexact filter within .get_indexer(). And the intersection() and union() uses these methods, so, in theory, one could pipe a tolerance argument through them (as well as for the other set operations). The work needs to be expanded a bit, though, as get_indexer_non_unique() needs the tolerance parameter, too, I think.

For xarray, though, I think we can work around backwards compatibility by having Dataset hold specialized subclasses of Index for floating-point data types that would have the needed changes to the Index class. We can have this specialized class have some default tolerance (say 100*finfo(dtype).resolution?), and it would have its methods use the stored tolerance by default, so it should be completely transparent to the end-user (hopefully). This way, xr.open_mfdataset() would "just work".

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/2217#issuecomment-399522595, or mute the thread https://github.com/notifications/unsubscribe-auth/ABKS1nkizKUQqKcLlnM0mn3rT_rqFfo5ks5t_SyGgaJpZM4UbV3q .