Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip not installed by default in aarch64-unknown-linux-gnu-noopt build #84

Closed
achimnol opened this issue Aug 19, 2021 · 2 comments
Closed
Labels
bug Something isn't working

Comments

@achimnol
Copy link

pip is not available right after extracting the aarch64 linux build of Python 3.9.6 (20210724).
Fortunately, it has the ensurepip with a bundled pip package, so I could workaround this by running python -m ensurepip once.
I'm reporting this to prevent other people from having the same gotcha.

From cpython-3.9.6-x86_64-unknown-linux-gnu-pgo-20210724T1424.tar.zst:

> find python/install -name '*pip*'
./lib/python3.9/test/test_pipes.py
./lib/python3.9/test/test_ensurepip.py
./lib/python3.9/pipes.py
./lib/python3.9/ensurepip
./lib/python3.9/ensurepip/_bundled/pip-21.1.3-py3-none-any.whl
./lib/python3.9/site-packages/pip-21.1.3-py3.9.egg
./lib/python3.9/site-packages/pip-21.1.3-py3.9.egg/pip
./bin/pip3
./bin/pip3.9
./bin/pip

From cpython-3.9.6-aarch64-unknown-linux-gnu-noopt-20210724T1424.tar.zst:

> find python/install -name '*pip*'
./lib/python3.9/test/test_pipes.py
./lib/python3.9/test/test_ensurepip.py
./lib/python3.9/pipes.py
./lib/python3.9/ensurepip
./lib/python3.9/ensurepip/_bundled/pip-21.1.3-py3-none-any.whl
@indygreg indygreg added the bug Something isn't working label Aug 20, 2021
@indygreg
Copy link
Collaborator

This is definitely an unintended bug. Thanks for reporting it.

This appears to only materialize in cross-builds. I think what's happening is because we use the host python to install pip it gets installed to the host python and not the target python we're cross-compiling to.

Things may get gnarly here. But I suspect we can coerce this into working without having to use an emulator to run the target Python. I'll try to look into this the next time I hack on this project.

indygreg added a commit that referenced this issue Aug 31, 2021
This commit refactors the install of pip and setuptools.

Previously, we installed both of these by invoking `setup.py` from the
host Python. This worked, but it was somewhat old school. And in the
case of cross-compiling, it installed the packages to the host Python,
not the target Python.

After this commit, we download wheels for both pip and setuptools and
the install invokes the host Python but runs the pip wheel to invoke
pip to install the packages. This is conceptually similar to how the
`ensurepip` module works.

The end result is that pip is used to install itself and setuptools
into the appropriate output directory. This works with both native
and cross builds.

Aside from minor changes to the directory layout, the end result is
the same as far as I can tell. But since cross-compiles now install
pip and setuptools correctly, this fixes #84.

Because pip now bootstraps self and this doesn't work on musl builds
without a patch, we had to move the patching of pip to before it
is invoked. We moved the patching of setuptools as well, because it
is related.
indygreg added a commit that referenced this issue Sep 3, 2021
This commit refactors the install of pip and setuptools.

Previously, we installed both of these by invoking `setup.py` from the
host Python. This worked, but it was somewhat old school. And in the
case of cross-compiling, it installed the packages to the host Python,
not the target Python.

After this commit, we download wheels for both pip and setuptools and
the install invokes the host Python but runs the pip wheel to invoke
pip to install the packages. This is conceptually similar to how the
`ensurepip` module works.

The end result is that pip is used to install itself and setuptools
into the appropriate output directory. This works with both native
and cross builds.

Aside from minor changes to the directory layout, the end result is
the same as far as I can tell. But since cross-compiles now install
pip and setuptools correctly, this fixes #84.

Because pip now bootstraps self and this doesn't work on musl builds
without a patch, we had to move the patching of pip to before it
is invoked. We moved the patching of setuptools as well, because it
is related.
@indygreg
Copy link
Collaborator

indygreg commented Sep 3, 2021

I force pushed the commit that fixes this off main because I'm running into issues with the latest Python 3.8 and 3.9 releases which introduced a build regression.

I plan to reintroduce this fix once I figure out the problems with the latest 3.8 and 3.9 versions.

@indygreg indygreg reopened this Sep 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants