issue_comments

I put a note in the gridded issue, but in short:

I think the way to get this is to use xarray in gridded, rather than to, well, reimplement gridded in xarray :-)

@dopplershift: indeed -- it's really the right way to go, but the community has been very, very slow to get there :(

BTW, I'm testing now, as as far as I can tell, pip has no problem up/downgrading setuptools. Even when installing xarray -- so the OP may really be facing a pip / setuptools / distro bug, that xarray should not try to accomodate :-)

However, some more testing shows that pip doesn't appear to try to upgrade setuptools to the version in setup_requires anyway. For example, I created a clean environment, installed setuptools version: 47.3.2, then edited the setup.cfg to:

install_requires = numpy >= 1.15 pandas >= 0.25 setuptools >= 41.2 # For pkg_resources setup_requires = setuptools >= 49 setuptools_scm

note that the installed setuptools meets the spec for install_requires, but not for setup_requires.

When I run the install, it works fine, and does not upgrade setuptools, or complain about it.

If I update the install_requies setuptools version, then it does upgrade it (successfully) on install.

So I suggest that we remove the setuptools requirement from setup_requires (or keep it where it is), and bump down the install_requires version to 30.3, or 37, if you really want.

This is kinda out of date:

https://stackoverflow.com/questions/58521386/should-setuptools-be-included-in-setup-requires-in-python

so maybe not true anymore, but it makes the point that setup_requires should not include setuptools itself -- as it won't be read until after setuptools has been imported anyway.

This may be a bit different with everything in setup.cfg, but I'm pretty sure that's still how it's usually run by pip -- that is setup.py is run, which imports setuptools, and then when setup() from setuptools is run, it reads the setup.cfg. So it's too late to check the setuptools version.

I'm making the distinction between the setuptools version used for building, and at run time.

I think you CAN expect people to have newish setuptools if they are building from source.

But that doesn't mean you have to require anyone running xarray to have a newish setuptools.

This is why there really should be separate packages for run time vs build time functionality, but that's out of our control.

I have tested that setuptools < 36.7 breaks setuptools-scm; the installed version becomes 0.0.0 which in turns breaks any other package that contains a minimum version check (namely, pandas).

Does it break anything if you use an older version only at run-time? I wouldn't think so.

It's not clear from the OP how they were installing -- i.e. from wheels or source, but if wheels, then pushing teh run time dependency back would fix it.

And setuptools 37 is from 20 Nov 2017, so probably safe :-)

well, I've had fine luck with simply using file and a relative path to get files. Though I suppose that would fail with a zipped package -- is xarray zipsafe otherwise?

Also, I appreciate the goal here, but in this case, it's only two files, though the CSS is pretty big.

but in any case, you have something that works, and you'd be hard pressed to find a python install that doesn't have setuptools -- so I'd say bump the Install-requires version back to an old one, and you're done.

NOTE: I'm still feeling the scars from setuptools nightmares bundling stand-alone applications from 15 years ago -- so I still don't like to use it at run-time -- but yeah, probably not relevant anymore :-)

and maybe revisit when you're no longer supporting 3.6

and when pyproject.toml was supported isn't even the point -- for whatever reason, it's still not very comon practice.

But when it comes to dependencies, setuptool is an odd one -- as it is used bioth to build teh pacakge and potentially at run time. I can rant about what a bad design that is, but it's a fact.

However, in the case of xarray, it seems to be used at runtime very sparingly:

(and then, only pkg_resources -- which really should be distributed by itself)

in init.py to get teh version number: __version__ = pkg_resources.get_distribution("xarray").version

I would argue that that is NOT a good way to manage your versioning, but certainly not the only way.

and in formatting_html.py

pkg_resources.resource_string("xarray", fname).decode("utf8")

I'm not sure I know what that actually does.

If it were me, I'd replace those and not have a setuptols run time dependency at all.

But in any case, unless pkg_resources has changed in recent years, you could probably have th run-time setuptools dependency be really old without problem.

That is separate from the install time dependency, which is really outside xarray anyway.

To summarize:

setuptools is used:

at built time: for that, xarray can use as recetn a version as you want.

at install time: this does overlap if folks are doing a source install -- but xarray has wheels up on pypi (and conda is pre-built) IN that case, it's up to teh user / pip to have setuptools if needed. When installing a wheel, I"m not sure setuptools is used at all, but in any case, pip does require it.

At run time: here is where the dependency matters: if I were you I"d get rid of the run time dependency, but if not, an old version should be fine here.

Looking at the setup,cfg file, I see:

install_requires = numpy >= 1.15 pandas >= 0.25 setuptools >= 41.2 # For pkg_resources setup_requires = setuptools >= 41.2 setuptools_scm note that you don't have to have the same version in both of these stanzas -- I think if you push the install_requires version back, you'll solve this problem without having to worry about building with old versions.

it made developers (particularly the less experienced ones) waste a considerable amount of effort retaining backwards compatibility with obsolete versions of the dependencies that nobody cared about.

that all depends on what you mean by "as-old-as-they-can-be" means -- to me, it means as old as they can be that have the features you need. IN this case, setuptools was being upgraded, even though xarray didn't need any of the new functionality.

What are you actually using for development? One option is to keep versions pinned to old versions until a developer wants a feature that requires a newer version -- then you update that one, if it's within your rolling window

But I can see that keeping track of what you need is tricky -- so the rolling window is a lot easier to manage

Honestly, the real problem here is that setuptools is both a build, install, and run-time dependency -- which is going to require special treatment.

As I said in the mailing list thread, if setuptools is only required for installation, it's not really a requirement at all. In fact, I'm pretty sure pip requires it, so it will always be there if pip is there.

But I see:

install_requires = numpy >= 1.15 pandas >= 0.25 setuptools >= 41.2 # For pkg_resources in setup.cfg -- so if you are using pkg_resources at run time, then you do need it here, in some version :-(

(the :-( is for pkg_resource being built in to setuptools -- it REALLY should be a separate package~!)

In regards to "don't ship what you don't test" -- if you take that philosophy, which is a good one, then you should be testing with-as-old-as-they-can-be versions of the dependencies anyway. which it looks like #4296 is doing :-)