issue_comments

scipy 1.1 and 1.2 break numpy 1.14 due to a conflict on openblas. We're stuck with 1.0 at least until the next numpy upgrade.

Ok, bumping pandas to >=0.24 and scipy to >=1.2

I think targeting 12 months of support for SciPy/pandas would be a reasonable goal.

On Wed, Oct 2, 2019 at 2:48 PM Maximilian Roos notifications@github.com wrote:

Overall this is awesome, thanks @crusaderky https://github.com/crusaderky

One minor question: do we need to support 24 months of pandas & scipy releases? These change their API faster than numpy, and I can't imagine there's a wide audience for people that a) need the very latest xarray and b) are using pandas released in 2017. I would vote to dramatically cut that, to something like the last two minor releases or 6 months. But could imagine two-three minor releases or 12 months being more palatable.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pydata/xarray/issues/3222?email_source=notifications&email_token=AAJJFVX4LXPFLJ7MOR6TN4LQMUJMHA5CNFSM4IMAETKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAGJQBI#issuecomment-537696261, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJJFVQBLJ7V5GRSCC6MXMLQMUJMHANCNFSM4IMAETKA .

Overall this is awesome, thanks @crusaderky

One minor question: do we need to support 24 months of pandas & scipy releases? These change their API faster than numpy, and I can't imagine there's a wide audience for people that a) need the very latest xarray and b) are using pandas released in 2017. I would vote to dramatically cut that, to something like the last two minor releases or 6 months. But could imagine two-three minor releases or 12 months being more palatable.

We should add this stuff to a doc page.

Thanks for putting this together -- this looks good to me!

We can revise if any of they cause us trouble. In particular, pandas has the most stability issues, so it might be a little annoying to try to maintain two years of support.

I slightly misinterpreted NEP-29. The above should be rectified:

pandas: 24 months (>=0.21) scipy: 24 months (>=1.0)

Revised proposal (in parenthesis: minimum version if this policy were to be applied right now):

• Python: NEP-29; 42 months (>=3.6)
• numpy: NEP-29; 24 months (>=1.14)
• pandas: 24 months (>=0.20)
• scipy: 24 months (>=0.19)
• NEP18 packages (sparse, pint, etc.): very latest available only, at least until the technology has reasonably matured across the whole ecosystem. This extends to dask when used in conjunction with any of these packages. numpy >=1.17. A very recent version of pandas may also be required if necessary.
• all other packages: 6 months or the latest available in the anaconda (not conda-forge) repository if it exists, whatever is older.

Note: patch versions are not pinned. This means that the minimum python version that is guaranteed to work is actually 3.6.7 or 3.6.9, not 3.6.0.

What are people's thoughts?

It may also be a cool idea to write a script that parses the requirements yml files and generates .rst documentation out of them.

@shoyer that sounds sensible, but only covers python and numpy. What about the rest?

it would be nice to try to follow the guidelines in NEP-29: https://numpy.org/neps/nep-0029-deprecation_policy.html

How does this sound as a rolling policy? - Python: latest 2 major versions published in official anaconda releases (as opposed to just the conda repository), plus the latest major version available version in conda-forge. Latest available minimum versions. This today translates to =3.6 and =3.7 (with no minor versions, so actually 3.6.7 and 3.7.4); 3.8 to be added as soon as conda-forge rebuilds its own stack with it. - numpy, pandas, scipy: latest major version at least 1 year old; latest available minor version (which may be newer than 1 year old). Today this translates to numpy=1.15, pandas=0.23, scipy=1.1. - all other non-NEP18 packages: 6 months old major versions, or the latest available in the anaconda (not conda-forge) repository if it exists, whatever is older - NEP18 packages (sparse, pint, etc.): very latest available only, at least until the technology has reasonably matured across the whole ecosystem. This extends to dask when used to wrap NEP18 arrays. numpy >=1.17. - In all cases: older versions than those stated by the above policy can remain officially supported indefinitely, but only as long as doing so can be achieved trivially and doesn't slow development down.

Today I tried downgrading dask in the py37 environment to dask=1.1.2, which is 6 months old...

Yikes, we should fix this.

It has not been hard to ensure that older version of NumPy and pandas work. The key thing is CI tests that verify things as PR goes in. If work-arounds for old versions are hard, then we bump minimum required dependencies.

Probably the best place to start would be adding a CI build with pinned versions of the oldest required versions that works today, and then looking if there are simple fixes that will let us downgrade project like dask. It's not worth a huge effort, but it's not very user friendly for everything to broken with old versions of optional dependencies either.