Skip to content

Revert removals introduced in v78.0.0 #4911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ jobs:
python: "3.10"
distutils: stdlib
runs-on: ${{ matrix.platform }}
continue-on-error: ${{ matrix.python == '3.14' }}
continue-on-error: ${{ matrix.python == '3.14' || matrix.python == 'pypy3.10' }}
# XXX: pypy seems to be flaky with unrelated tests in #6345
env:
SETUPTOOLS_USE_DISTUTILS: ${{ matrix.distutils || 'local' }}
timeout-minutes: 75
Expand Down
2 changes: 2 additions & 0 deletions newsfragments/4911.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Postponed removals of deprecated dash-separated and uppercase fields in ``setup.cfg``.
All packages with deprecated configurations are advised to move before 2026.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! But when you eventually do restore this behavior, could we please get the option to opt out?

Some packages are simply old & abandoned, and there's not much that consumers can do when trying to install a third-party package (as the many comments on #4910 would indicate!)

Copy link
Contributor Author

@abravalheri abravalheri Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The option is to pin setuptools via the PIP_CONSTRAINT environment variable (or the equivalent in UV).

Some packages are simply old & abandoned, and there's not much that consumers can do when trying to install a third-party package (as the many comments on #4910 would indicate!)

Please note that this only affects packages that do not publish a wheel to PyPI or other package index. Unless it is a module with native binary extensions, the recommendation of publishing wheels is a bit old already.

All things considered this is likely a symptom of something bigger, more problematic, and the package you depend upon has been on borrowed time for a while. If you depend on packages that do not receive any support you might be subject to all sorts of risks including security risks. In that case there are a couple of things that can be done during the extended period:

  • Organise among other users contributions (or even a fork) for maintenance.
  • Fund the original developers for maintenance.
  • Find a suitable replacement.

Copy link

@justinoboyle justinoboyle Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may well all be true. But why not show a loud warning for a while before removing totally? The reality is that we try to vet our packages carefully, but still woke up to Monday morning with lots of broken build processes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the issue does also affect all Linux distributions and downstream vendors that rebuild wheels from sources. Linux distributions like Debian, Fedora, and Gentoo do not use wheels. They take sdists and rebuild wheels.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

…and given the scale of this issue, it will mean we won't be able upgrade setuptools for months. Or to put it more precisely, once again the burden of fixing all the fallout will be on downstreams, who spend their whole days putting out fires for free.

Copy link

@eskil eskil Mar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely sympathize with the goal of getting all dependencies to do the right thing and being able to simplify the setup tools code. But I agree with @danielcarcich. From a robustness principle, this seems very unfortunate.

While this is easy to fix in smaller repos, many large orgs deal with a ton of unfortunately old and unmaintained packages. Yes that's a fault of them (I'm in one), but here we are.

I hope to see this reverted and thanks to @abravalheri for starting this pull-request preemptively while having the discussion. (apologies is this conversation should be in #4910 instead)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The option is to pin setuptools via the PIP_CONSTRAINT environment variable (or the equivalent in UV).

Some packages are simply old & abandoned, and there's not much that consumers can do when trying to install a third-party package (as the many comments on #4910 would indicate!)

Please note that this only affects packages that do not publish a wheel to PyPI or other package index. Unless it is a module with native binary extensions, the recommendation of publishing wheels is a bit old already.

All things considered this is likely a symptom of something bigger, more problematic, and the package you depend upon has been on borrowed time for a while. If you depend on packages that do not receive any support you might be subject to all sorts of risks including security risks. In that case there are a couple of things that can be done during the extended period:

  • Organise among other users contributions (or even a fork) for maintenance.
  • Fund the original developers for maintenance.
  • Find a suitable replacement.

A build tool needs to aim to interact with as many versions of libraries from the ecosystem as possible, not just the latest versions that adhere to modern best practices. Thanks for all the hard work maintaining a core utility and for responding quickly to the response today.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I say this as someone who has contributed a patch to a popular community package to remediate a CVE, and maintained an internal fork waiting for a new release. My PR was accepted, but no release yet for over a year. Ok, not that big of a contribution, but you can't accuse me of never having contributed.

It will be 1. undifferentiated heavy lifting and 2. utter chaos to maintain internal forks or coordinate community forks for every single transitive dependency in our codebase, especially as others also have the dependencies as well.

If Setuptools does this again in 2026, it will be a repeat of this. How many warnings went unheeded this time? What are the chances those warnings will be heeded in the year to come? It will take years, not months, to untangle.

If the plan is to try this again in 2026, the package to be forked will be setuptools.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While the contributions of Setuptools maintainers are very appreciated, I think it's important to realize that the Setuptools API has become a load-bearing component of the distribution infrastructure of many legacy Python packages that can no longer be easily updated, yet are widely needed for the foreseeable future. As such, I think it's simply not possible to introduce a change like this one without unbearable levels of disruption. A more finessed approach (like selectively treating these errors as warnings based on signals like the age of the package release, etc.) is necessary. And also a rethink of how to restrict and compartmentalize the packaging API surface so build dependencies for old packages can be bundled with them for future issues like these. PEP 517 takes us most of the way to where we want to be, but more work is needed I think.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Setuptools does this again in 2026, it will be a repeat of this. How many warnings went unheeded this time? What are the chances those warnings will be heeded in the year to come? It will take years, not months, to untangle.

Concur. Yes, some projects will fix the issue in the meantime (and some did in response to this). But there are still a fair number of basically-unmaintained packages out there, and pushing this back by a year won't solve that.

If the plan is to try this again in 2026, the package to be forked will be setuptools.

Agree. I'm not saying that's how it should be, but that's a potential outcome if this still goes forward.

I know from personal experience how stressful and thankless a job it can be maintaining open source tools that hundreds of thousands or millions of devs depend on.

One of the key responsibilities from that is knowing that if you introduce a breaking change -- no matter HOW well you pre-communicate it -- you're potentially breaking tons of people, and some of them will simply settle on an old version and refuse to upgrade from then on. If enough of them are upset the project gets forked. The usual escape valve to prevent this is offering an override option to restore the previous behavior. It sucks, it can be maddening being so restricted in the ability to move beyond past technical/feature mistakes. But you learn to pick and choose your battles, and something like underscores-vs-hyphens usually isn't worth the fight.

TL;DR: "With great power comes great responsibility" is the mantra for critical open source projects -- writ large in all-caps bold.

28 changes: 24 additions & 4 deletions setuptools/dist.py
Original file line number Diff line number Diff line change
Expand Up @@ -626,21 +626,41 @@ def _enforce_underscore(self, opt: str, section: str) -> str:
if "-" not in opt or self._skip_setupcfg_normalization(section):
return opt

raise InvalidConfigError(
underscore_opt = opt.replace('-', '_')
affected = f"(Affected: {self.metadata.name})." if self.metadata.name else ""
SetuptoolsDeprecationWarning.emit(
f"Invalid dash-separated key {opt!r} in {section!r} (setup.cfg), "
f"please use the underscore name {opt.replace('-', '_')!r} instead."
f"please use the underscore name {underscore_opt!r} instead.",
f"""
Usage of dash-separated {opt!r} will not be supported in future
versions. Please use the underscore name {underscore_opt!r} instead.
{affected}
""",
see_docs="userguide/declarative_config.html",
due_date=(2026, 3, 3),
# Warning initially introduced in 3 Mar 2021
)
return underscore_opt

def _enforce_option_lowercase(self, opt: str, section: str) -> str:
if opt.islower() or self._skip_setupcfg_normalization(section):
return opt

raise InvalidConfigError(
lowercase_opt = opt.lower()
affected = f"(Affected: {self.metadata.name})." if self.metadata.name else ""
SetuptoolsDeprecationWarning.emit(
f"Invalid uppercase key {opt!r} in {section!r} (setup.cfg), "
f"please use lowercase {opt.lower()!r} instead."
f"please use lowercase {lowercase_opt!r} instead.",
f"""
Usage of uppercase key {opt!r} in {section!r} will not be supported in
future versions. Please use lowercase {lowercase_opt!r} instead.
{affected}
""",
see_docs="userguide/declarative_config.html",
due_date=(2026, 3, 3),
# Warning initially introduced in 6 Mar 2021
)
return lowercase_opt

def _skip_setupcfg_normalization(self, section: str) -> bool:
skip = (
Expand Down
23 changes: 17 additions & 6 deletions setuptools/tests/config/test_setupcfg.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@

from setuptools.config.setupcfg import ConfigHandler, Target, read_configuration
from setuptools.dist import Distribution, _Distribution
from setuptools.errors import InvalidConfigError
from setuptools.warnings import SetuptoolsDeprecationWarning

from ..textwrap import DALS
Expand Down Expand Up @@ -423,7 +422,7 @@ def test_not_utf8(self, tmpdir):
pass

@pytest.mark.parametrize(
("error_msg", "config"),
("error_msg", "config", "invalid"),
[
(
"Invalid dash-separated key 'author-email' in 'metadata' (setup.cfg)",
Expand All @@ -434,6 +433,7 @@ def test_not_utf8(self, tmpdir):
maintainer_email = [email protected]
"""
),
{"author-email": "[email protected]"},
),
(
"Invalid uppercase key 'Name' in 'metadata' (setup.cfg)",
Expand All @@ -444,14 +444,25 @@ def test_not_utf8(self, tmpdir):
description = Some description
"""
),
{"Name": "foo"},
),
],
)
def test_invalid_options_previously_deprecated(self, tmpdir, error_msg, config):
# this test and related methods can be removed when no longer needed
def test_invalid_options_previously_deprecated(
self, tmpdir, error_msg, config, invalid
):
# This test and related methods can be removed when no longer needed.
# Deprecation postponed due to push-back from the community in
# https://github.com/pypa/setuptools/issues/4910
fake_env(tmpdir, config)
with pytest.raises(InvalidConfigError, match=re.escape(error_msg)):
get_dist(tmpdir).__enter__()
with pytest.warns(SetuptoolsDeprecationWarning, match=re.escape(error_msg)):
dist = get_dist(tmpdir).__enter__()

tmpdir.join('setup.cfg').remove()

for field, value in invalid.items():
attr = field.replace("-", "_").lower()
assert getattr(dist.metadata, attr) == value


class TestOptions:
Expand Down
Loading