[BUG] Difference among distutils.sysconfig.get_python_inc()

[wqh17101]

setuptools version

65.3.0

Python version

3.9.2

OS

Linux

Additional environment information

My env is in a docker.

Description

Hi there, i find that setuptools 65.3.0 hack the distutils. I want to get the include dir of python
And i found that

before install setuptools

python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'

and get /opt/aaa/python/python-3.9.2/include/python3.9

after install that

python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'

and get /devcloud/ws/sirFh/workspace/j_VWBQ8FGA/aaa_python/third_build/python/include/python3.9

/opt/aaa/python/python-3.9.2/include/python3.9 is the right anwser.
And /devcloud/ws/sirFh/workspace/j_VWBQ8FGA/aaa_python/third_build/python/include/python3.9 does not exist in my docker or even my machine

Expected behavior

Get the right include dir

How to Reproduce

import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())

Output

python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'

[abravalheri]

Hi @wqh17101, could you try running this with https://github.com/pypa/distutils instead of the distutils that comes with the standard library?

The distutils library is marked for removal from the stdlib and is currently being maintained/receiving updates in https://github.com/pypa/distutils. When you install & import setuptools, it will automatically redirect the imports to use the latest updated version.

Regarding the reproducer, if we need to investigate the issue deeper we will need a bit more information than linux/docker. Is there any chance you can share a minimal Dockerfile that reproduces the error? You can find some tips on how to right helpful reproducers in this post from stackoverflow.

[wqh17101]

So that you mean this is the bug of https://github.com/pypa/distutils ?@abravalheri

[abravalheri]

So that you mean this is the bug of https://github.com/pypa/distutils ?

I don't think we can be categorical without a proper reproducer. Could you have a look on the 2 items I proposed in the previous comment?

could you try running this with https://github.com/pypa/distutils instead of the distutils that comes with the standard library?

Have you tried this one? What is the outcome?

Is there any chance you can share a minimal Dockerfile that reproduces the error? You can find some tips on how to right helpful reproducers in this post from stackoverflow.

Could you please share with us a reproducer?

[wqh17101]

could you try running this with https://github.com/pypa/distutils instead of the distutils that comes with the standard library?

Have you tried this one? What is the outcome?

How to install this one? As you say, When you install & import setuptools, it will automatically redirect the imports to use the latest updated version. I believe it use the latest one of distutils.

Sorry,Hard to make a reproducer for me.

[abravalheri]

How to install this one?

If you clone https://github.com/pypa/distutils and run the experiment from within the folder, that should do the trick.

Sorry,Hard to make a reproducer for me.

This makes it really difficult to investigate/help.

[wqh17101]

How to install this one?

If you clone https://github.com/pypa/distutils and run the experiment from within the folder, that should do the trick.

get the same result as setuptools,/devcloud/ws/sirFh/workspace/j_VWBQ8FGA/aaa_python/third_build/python/include/python3.9

Sorry,Hard to make a reproducer for me.

This makes it really difficult to investigate/help.

Maybe you can test in any docker env?

[abravalheri]

Maybe you can test in any docker env?

I cannot observe the described behaviour in the following experiment:

> docker run --rm -it python:3.9.2 /bin/bash

python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'
# /usr/local/include/python3.9

python -m pip install -U pip setuptools
# ...
# Successfully installed pip-22.2.2 setuptools-65.3.0

python -c 'import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'
# /usr/local/include/python3.9

python -c 'import setuptools; import distutils.sysconfig; print(distutils.sysconfig.get_python_inc())'
# /usr/local/include/python3.9

We will probably need a minimal reproducer to proceed with the investigation.

[wqh17101]

@abravalheri I found the reason. pypa/distutils#179

[zjp]

pypa/distutils#178 (comment)

I opened a similar issue in the distutils repository. I have a reproducible case with ChimeraX.

[zjp]

If you prefer not to install ChimeraX, you can pull our docker container rbvi/chimerax:latest.

[root@3be7e637a87b ChimeraX]# ChimeraX.app/bin/ChimeraX -c 'from distutils import sysconfig; print(sysconfig.get_python_inc())'
/ChimeraX/ChimeraX.app/include/python3.9

[root@3be7e637a87b ChimeraX]# ChimeraX.app/bin/ChimeraX -m pip install -U setuptools
Requirement already satisfied: setuptools in ./ChimeraX.app/lib/python3.9/site-packages (65.1.1)
Collecting setuptools
  Downloading setuptools-65.5.0-py3-none-any.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 12.9 MB/s eta 0:00:00
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 65.1.1
    Uninstalling setuptools-65.1.1:
      Successfully uninstalled setuptools-65.1.1
Successfully installed setuptools-65.5.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip available: 22.2.2 -> 22.3
[notice] To update, run: /ChimeraX/ChimeraX.app/bin/ChimeraX -m pip install --upgrade pip

[root@3be7e637a87b ChimeraX]# ChimeraX.app/bin/ChimeraX -c 'from distutils import sysconfig; print(sysconfig.get_python_inc())'
/ChimeraX/build/include/python3.9

As you can see, the header locations change from the bundled Python's location to wherever they were when that Python was built.