issue_comments

@nbren12 - the key difference for our micro-packages is that the primary maintainer is me, not my grad students, and I'm not going anywhere for now. 😉

I still agree that there is probably a better way to organize all of this. Just trying to share our perspective as an xarray-centric small research group.

The approach we have been taking is to develop "micro-packages". We currently have three: - xgcm - for finite volume cell operations on top of xarray DataArrays - xrft - for coordinate-aware Fourier transforms of Xarray DataArrays - xhistogram - (this one is brand new) - for multidimensional histograms applied along specified axes

These packages share some common design principles. In particular, they are all fully lazy and dask-friendly, meaning that we can apply them to very large datasets (which is the main focus in our group). By keeping the packages small, they are more maintainable. Xgcm and Xrft probably have O(3) active contributors, primarily myself and grad students in my group. Small, but significantly different from 1. We use these packages heavily in everyday scientific work, so I know they are useful.

I would love to combine forces on a larger effort. However, we have limited time and effort. For now, however, this situation doesn't seem too bad. It's kind of compatible with what @teoliphant was suggesting in his comment 1 above. I'm not sure that some mega xarray-contrib package would have critical mass to be sustainable either.

Just to add to the mix, we have our own package for spectra! https://xrft.readthedocs.io/en/latest/

On Apr 4, 2019, at 5:04 PM, Noah D Brenowitz notifications@github.com wrote:

Thanks @rabernat that awesome list looks pretty awesome.

However, I would still advocate for a more centralized approach to this problem. For instance, the NCL has a huge library of contributed functions which they distribute along with the code. By now, I am sure that xarray users have basically reimplemented equivalents to all of these functions, but without a centralized home it is still too difficult to find or contribute new codes.

For instance, I have a useful wrapper to scipy.ndimage that I use all the time, but it seems overkill to release/support a whole package for this one module. I would be much more likely to contribute a PR to a community run repository. I am also much more likely to use such a repo.

I would be more than willing to volunteer for such an effort, but I think it needs to involve multiple people. Various individuals have tried to make such repos on their own, but none seem to have reached critical mass. For example, https://github.com/crusaderky/xarray_extras https://github.com/fujiisoup/xr-scipy I think there should be multiple maintainers, so that if one person drops out, there still appears to be activity on the repo.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

FYI, we have started https://github.com/pangeo-data/awesome-open-climate-science. It is not xarray specific, but contains many xarray-related packages. Please contribute!