Adopt the “time window-based” policy for support of Python versions from Spec-0

[tobiasdiez]

📚 Description

Add a short paragraph to the developer guide explaining the Python support schedule. Previously discussed in #30384 and on the mailing list. Closes #30384.

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

[mkoeppe]

In case you didn't see it - https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/x3qOdPB5BQAJ

[tobiasdiez]

No, I've seen your comments. But what you propose essentially comes down to NEP29 + half a year or so, which just feels random. Also your work on the modularization is helping to promote that sage is used more actively in other python packages, so it gets more and more important to sync with the community standards.

[mkoeppe]

I've seen your comments. But what you propose [...]

Tobias, I'm not "proposing" it. I'm describing our actual practice.

Also your work on the modularization is helping to promote that sage is used more actively in other python packages,
so it gets more and more important to sync with the community standards.

You don't need to explain this to me; but it has not happened yet.

[dimpase]

our actual practice is not ideal, on many fronts.

[mkoeppe]

General comments like this can hardly be a useful contribution to this discussion.

[dimpase]

no, you are declining to proceed with this PR on the same kind of grounds as my comment is based upon.

[mkoeppe]

Re-read https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/x3qOdPB5BQAJ; it has a detailed discussion, which cannot be dismissed by generalities. If you want to discuss -- then discuss, don't dismiss.

[dimpase]

you can reread my comment in that link, too.
We can have a vote on this...

I don't understand why Sage must be trailing behind a number of its upstream components.

[mkoeppe]

I don't understand why Sage must be trailing behind a number of its upstream components.

Well, I have explained that it does not. No package relevant for Sage has already dropped Python 3.8 in a released version or will before July.

[mkoeppe]

By the way, the numpy update to latest is waiting for review in #35081.

[dimpase]

Scipy is definitely dropping 3.8 soon, and they say they do follow NEP 29.
"When dropping support for older Python versions, SciPy takes guidance from NEP 29".
cf. https://docs.scipy.org/doc/scipy/dev/toolchain.html

Are we following NEP 29, or not?

[mkoeppe]

Scipy is definitely dropping 3.8 soon, and they say they do follow NEP 29.

I explained the specifics in https://groups.google.com/g/sage-devel/c/j1cwbTU8aOU/m/x3qOdPB5BQAJ

[mkoeppe]

Are we following NEP 29, or not?

No, we don't have this policy. The present PR (based on the Trac ticket that I opened in 2020, #30384), proposes to change that.

[tobiasdiez]

In order to bring this back onto a good discussion platform, let me summarize your arguments for not adopting NEP 29:

• It is not the current practice of how sage operates.
• NEP 29 is mostly relevant for packages that are used by other packages, but sage is only a downstream application.
• No one is forcing us to drop support of a python version until scipy/numpy or other dependencies are doing this.
• Some linux distributions (that are still having lts) are coming with preinstalled python that is not compatible with NEP 29.

Is this correct so far @mkoeppe? Did I miss an argument or misunderstood you?

My response to these points is:

• While there is definitely value in having an organically grown policy, I think its important from time to time question if it still fits well with the current situation. From a users perspective, one could argue that they are used to a certain drop schedule for python support in sage, but since this was never officially documented or communicated (please correct me if I'm wrong), adopting NEP29 provides more clarity for users on what they can expect and aligns sage with the common practice of other packages that they may use.

• Sure, NEP 29 is mainly handy for downstream packages as they can expect what versions are supported. But (as I don't have to tell you) the modularization effort has the goal to make sage useful as an upstream package. So you can see this push towards NEP29 as a helpful step for modularization/pip installable packages which is future proof.

• The disadvantages of taking Linux distributions as reference is addressed in the NEP itself very well in my opinion: https://numpy.org/neps/nep-0029-deprecation_policy.html#default-version-on-linux-distribution. In particular, I completely agree with

  By following the versions supported by major Linux distributions, we are giving up technical control of our projects to external organizations that may have different motivations and concerns than we do.

  The same problems occur in my opinion when you take dependencies as your reference point. Say numpy changes their drop schedule to only support the last python version, does sage then follows this as well?

Finally, what makes sage so fundamentally different to scipy that its users should expect sage to support python versions released in the last 48 months, while scipy users are totally fine with support for releases in the last 42 months? Especially given that its very easy to install a new python version (pyenv, conda, ...). From experience, its definitely easier than installing sage itself ;-).

In general, we have two competing view points:

• As a user, I want sage to work with all python versions (since then I don't have to worry about the python version I have installed)
• As a sage developer, I want to only support the latest python version (since then I don't have to test on different versions, can use the latest language features, can reduce the technical dept)

Obviously, there is no objective line where these competing wishes are perfectly balanced. But I think its usually a good idea to align with established policies in the field in such cases, and NEP29 is the defacto most popular one.

[mkoeppe]

• From a users perspective, one could argue that they are used to a certain drop schedule for python support in sage, but since this was never officially documented or communicated (please correct me if I'm wrong), adopting NEP29 provides more clarity for users on what they can expect and aligns sage with the common practice of other packages that they may use.

That's really far fetched. Our users have never asked us anything like that.

[mkoeppe]

So you can see this push towards NEP29 as a helpful step for modularization/pip installable packages

I've already responded to this – yes, I'm aware of it; yes, making (portions of) the Sage library suitable as upstream for other packages is very much the point of the modularization; but no, it is not relevant yet.

[mkoeppe]

we are giving up technical control of our projects to external organizations [...]
The same problems occur in my opinion when you take dependencies as your reference point. Say numpy changes their drop schedule to only support the last python version, does sage then follows this as well?

This is getting dialectical.

[tobiasdiez]

• From a users perspective, one could argue that they are used to a certain drop schedule for python support in sage, but since this was never officially documented or communicated (please correct me if I'm wrong), adopting NEP29 provides more clarity for users on what they can expect and aligns sage with the common practice of other packages that they may use.

That's really far fetched. Our users have never asked us anything like that.

What about seeing this as an experiment: we follow NEP29 for this release (i.e. drop python 3.8 support), and if there is a huge backslash by the community we change it to NEP29 + half a year?

[tobiasdiez]

So you can see this push towards NEP29 as a helpful step for modularization/pip installable packages

I've already responded to this – yes, I'm aware of it; yes, making (portions of) the Sage library suitable as upstream for other packages is very much the point of the modularization; but no, it is not relevant yet.

Yes, you made this point already. But dismissing something as not relevant without giving reasons is neither constructive nor helpful.

[mkoeppe]

But dismissing something as not relevant without giving reasons

Please, Tobias, that's inappropriate. I have already explained the reason. It's irrelevant now (for 10.0) because the modularization has not happened yet, so there's no real benefit; only an imaginary benefit.

[mkoeppe]

What about seeing this as an experiment

Not interested.

[dimpase]

there are obvious benefits in becoming a community member.

Saying that historically Sage was not a member does not mean it has to continue to be so, and if you disagree, you can either call a vote on it, or just fork Sage and be your own boss.

[github-actions]

Documentation preview for this PR (built with commit 23d5360; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[vbraun]

NEP 29 is now superseded by https://scientific-python.org/specs/spec-0000/ so at least that needs updating

• Maybe not mention any particular dates (ages like milk), just link to external documents for time tables
• I don't really see where the disagreement here is, can you just agree on a good enough formulation

[mkoeppe]

For this one, we had started a longer discussion on sage-devel.
There's a page at https://github.com/sagemath/sage/wiki/NEP-29:-Python-version-strategy with many details.

This needs to be resolved by bringing this back to sage-devel, definitely not by just merging a PR.

[dimpase]

I'd rather delegate the task of deciding this to @vbraun

Discussions on any topic Matthias has an opinion deviating from other's people opinions lead to a riot lately.

[tobiasdiez]

NEP 29 is now superseded by https://scientific-python.org/specs/spec-0000/ so at least that needs updating

* Maybe not mention any particular dates (ages like milk), just link to external documents for time tables

* I don't really see where the disagreement here is, can you just agree on a good enough formulation

Thanks a lot for these suggestions! I've now changed this PR to use Spec-0 and referred to Spec-0 for the specific dates.
Spec-0 is also very clear in its formulation (e.g. it explicitly uses "drop schedule") which should resolve the concerns raised in the past about the NEP29 version. Thus, I think we can remove the "disputed" flag.