issue_comments: 523960862

I have a draft solution in #3239. It adds a new mode called "sensible" that acts like "all" when the concat dimension doesn't exist in the dataset and acts like "minimal" when the dimension is present. We can decide whether this is the right way i.e. add a new mode but the more fundamental problem is below.

The issue is dealing with variables that should not be concatentated in "minimal" mode (e.g. time-invariant non dim coords when concatenating in time). In this case, we want to skip the equality checks in _calc_concat_over. This is a common reason for poor open_mfdataset performance.

I thought the clean way to do this would be to add the compat kwarg to concat and then add compat='override' since the current behaviour is effectively compat='equals'.

However, merge takes compat too and concat and merge support different compat arguments at present. This makes it complicated to easily thread compat down from combine or open_mfdataset without adding concat_compat and merge_compat which is silly.

So do we want to support all the other compat modes in concat? Things like broadcast_equals or no_conflicts are funny because they're basically merge operations and it means concat acts like both stack, concat and merge. OTOH if you have a set of variables with the same name from different datasets and you want to pick one of those (i.e. no concatenation), then you're basically doing merge anyway. This would require some refactoring since concat assumes the first dataset is a template for the rest.

@shoyer What do you think?