build_meta sdist directory delegate to --dist-dir #1481

gaborbernat · 2018-09-14T14:04:19Z

This avoids a lot of current bugs (e.g. sdists are always generated in dist, and at end moved out; however dist folder remains around - so if e.g. the version changes you'll get two sdists now); no need to move around just build in the requested location. Discovered this while trying to use this backend with tox's PEP-517/8.

pganssle · 2018-09-14T14:45:16Z

I think the code coverage failure is just because you have simplified the code and the total number of covered lines went down, so I think we can ignore it.

pganssle · 2018-09-14T14:46:08Z

@gaborbernat Can you add some tests for any of the bugs that this fixes?

gaborbernat · 2018-09-14T14:51:20Z

I can try 👍 there are multiple bugs though.

pganssle · 2018-09-14T14:51:51Z

@gaborbernat "This fixes too many bugs" is a pretty nice problem to have. ;)

benoit-pierre · 2018-09-14T15:37:02Z

@pganssle: what's the deal with netlify? I can't reproduce the issue locally when running tox -e docs with Python 2.7.15.

gaborbernat · 2018-09-14T17:12:49Z

@pganssle added two of them

pganssle

I looked over the code and the tests and it looks good to me, but I'm going to defer the merge to either @jaraco or @benoit-pierre because I don't feel like I totally understand the implications here, having never used this particular option.

If this is time-critical and the other maintainers don't have time for it, let me know and I'll look into the details of how this is used a little deeper to give myself the confidence to push the "merge" button.

It still does need a changelog entry, however.

pganssle · 2018-09-16T17:13:30Z

@gaborbernat I rebased this against master because I believe the netlify problem is fixed in master, but that obviously wiped out your signatures. Here's my alias for "rebase-and-sign-all-commits" if you want to use it to do a new rebase.

gaborbernat · 2018-09-16T21:17:59Z

@pganssle added the changelog. I would say this is fairly critical as without it the build frontend (in my case tox) is troublesome to use. E.g. the user needs to manually clean the dist folder between version changes for a package (which when using setuptools-scm is at every commit).

gaborbernat · 2018-09-18T07:45:26Z

@benoit-pierre @jaraco @pganssle can we get a merge and release on this? thank you very much.

@pfmoore did you run into this issue while implementing PEP-517 for pip?

pfmoore · 2018-09-18T07:47:22Z

No, it hasn't affected me. Pip doesn't use the build_sdist hook, though, only build_wheel.

gaborbernat · 2018-09-18T07:50:08Z

build_wheel I think has the same issue; though to be fair if you start with a clean folder the issue does not manifest (which pip does I assume). The problem arrises more if you do subsequent builds (e.g. dist already exists and contains an sdist - especially if you change some metadata - such as version - to generate a differently named sdist). Or you specify the dist as the build dist directory.

pfmoore · 2018-09-18T08:01:52Z

Yeah, I don't think we do that (although tbh we use https://github.com/pypa/pep517 so the details are hidden in there). Does tox use its own wrapper code rather than using that library? (I wouldn't be surprised if it did, the pep517 library isn't particularly well publicised)

gaborbernat · 2018-09-18T08:11:00Z

I was aware of that (I pointed it out for you 😁) but given we need a subset of that, decided not to pull it in.

pfmoore · 2018-09-18T08:35:57Z

lol yes you did. Looks like old age is getting to my brain cells 😄

jaraco · 2018-09-18T13:07:56Z

Looks good to me. Let's give it a spin.

jaraco · 2018-09-18T13:09:02Z

Releasing as 40.4.0.

gaborbernat · 2018-09-18T14:26:17Z

Thanks will validate this by tomorrow.

gaborbernat · 2018-09-18T20:55:44Z

Managed to validate it, works fine for me 👍

### Update [tox](https://pypi.org/project/tox) from **3.2.1** to **3.4.0**. <details> <summary>Changelog</summary> ### 3.4.0 ``` ------------------ Bugfixes ^^^^^^^^ - add ``--exists-action w`` to default pip flags to handle better VCS dependencies (`pip documentation on this <https://pip.pypa.io/en/latest/reference/pip/exists-action-option>`_) - by :user:`gaborbernat` (`503 <https://github.com/tox-dev/tox/issues/503>`_) - instead of assuming the Python version from the base python name ask the interpreter to reveal the version for the ``ignore_basepython_conflict`` flag - by :user:`gaborbernat` (`908 <https://github.com/tox-dev/tox/issues/908>`_) - PEP-517 packaging fails with sdist already exists, fixed via ensuring the dist folder is empty before invoking the backend and `pypa/setuptools 1481 <https://github.com/pypa/setuptools/pull/1481>`_ - by :user:`gaborbernat` (`1003 <https://github.com/tox-dev/tox/issues/1003>`_) Features ^^^^^^^^ - add ``commands_pre`` and ``commands_post`` that run before and after running the ``commands`` (setup runs always, commands only if setup suceeds, teardown always - all run until the first failing command) - by :user:`gaborbernat` (`167 <https://github.com/tox-dev/tox/issues/167>`_) - ``pyproject.toml`` config support initially by just inline the tox.ini under ``tool.tox.legacy_tox_ini`` key; config source priority order is ``pyproject.toml``, ``tox.ini`` and then ``setup.cfg`` - by :user:`gaborbernat` (`814 <https://github.com/tox-dev/tox/issues/814>`_) - use the os environment variable ``TOX_SKIP_ENV`` to filter out tox environment names from the run list (set by ``envlist``) - by :user:`gaborbernat` (`824 <https://github.com/tox-dev/tox/issues/824>`_) - always set ``PIP_USER=0`` (do not install into the user site package, but inside the virtual environment created) and ``PIP_NO_DEPS=0`` (installing without dependencies can cause broken package installations) inside tox - by :user:`gaborbernat` (`838 <https://github.com/tox-dev/tox/issues/838>`_) - tox will inject some environment variables that to indicate a command is running within tox: ``TOX_WORK_DIR`` env var is set to the tox work directory, ``TOX_ENV_NAME`` is set to the current running tox environment name, ``TOX_ENV_DIR`` is set to the current tox environments working dir - by :user:`gaborbernat` (`847 <https://github.com/tox-dev/tox/issues/847>`_) - While running tox invokes various commands (such as building the package, pip installing dependencies and so on), these were printed in case they failed as Python arrays. Changed the representation to a shell command, allowing the users to quickly replicate/debug the failure on their own - by :user:`gaborbernat` (`851 <https://github.com/tox-dev/tox/issues/851>`_) - skip missing interpreters value from the config file can now be overridden via the ``--skip-missing-interpreters`` cli flag - by :user:`gaborbernat` (`903 <https://github.com/tox-dev/tox/issues/903>`_) - keep additional environments config order when listing them - by :user:`gaborbernat` (`921 <https://github.com/tox-dev/tox/issues/921>`_) - allow injecting config value inside the ini file dependent of the fact that we're connected to an interactive shell or not - by :user:`gaborbernat` (`947 <https://github.com/tox-dev/tox/issues/947>`_) - do not build sdist if skip install is specified for the envs to be run - by :user:`gaborbernat` (`974 <https://github.com/tox-dev/tox/issues/974>`_) - when verbosity level increases above two start passing through verbosity flags to pip - by :user:`gaborbernat` (`982 <https://github.com/tox-dev/tox/issues/982>`_) - when discovering the interpreter to use check if the tox host Python matches and use that if so - by :user:`gaborbernat` (`994 <https://github.com/tox-dev/tox/issues/994>`_) - ``-vv`` will print out why a virtual environment is re-created whenever this operation is triggered - by :user:`gaborbernat` (`1004 <https://github.com/tox-dev/tox/issues/1004>`_) Documentation ^^^^^^^^^^^^^ - clarify that ``python`` and ``pip`` refer to the virtual environments executable - by :user:`gaborbernat` (`305 <https://github.com/tox-dev/tox/issues/305>`_) - add Sphinx and mkdocs example of generating documentation via tox - by :user:`gaborbernat` (`374 <https://github.com/tox-dev/tox/issues/374>`_) - specify that ``setup.cfg`` tox configuration needs to be inside the ``tox:tox`` namespace - by :user:`gaborbernat` (`545 <https://github.com/tox-dev/tox/issues/545>`_) ``` ### 3.3.0 ``` ------------------ Bugfixes ^^^^^^^^ - fix ``TOX_LIMITED_SHEBANG`` when running under python3 - by :user:`asottile` (`931 <https://github.com/tox-dev/tox/issues/931>`_) Features ^^^^^^^^ - `PEP-517 <https://www.python.org/dev/peps/pep-0517/>`_ source distribution support (create a ``.package`` virtual environment to perform build operations inside) by :user:`gaborbernat` (`573 <https://github.com/tox-dev/tox/issues/573>`_) - `flit <https://flit.readthedocs.io>`_ support via implementing ``PEP-517`` by :user:`gaborbernat` (`820 <https://github.com/tox-dev/tox/issues/820>`_) - packaging now is exposed as a hook via ``tox_package(session, venv)`` - by :user:`gaborbernat` (`951 <https://github.com/tox-dev/tox/issues/951>`_) Miscellaneous ^^^^^^^^^^^^^ - Updated the VSTS build YAML to use the latest jobs and pools syntax - by :user:`davidstaheli` (`955 <https://github.com/tox-dev/tox/issues/955>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/tox - Changelog: https://pyup.io/changelogs/tox/ - Docs: https://tox.readthedocs.org/ </details> ### Update [Sphinx](https://pypi.org/project/Sphinx) from **1.8.0** to **1.8.1**. <details> <summary>Changelog</summary> ### 1.8.1 ``` ===================================== Incompatible changes -------------------- * LaTeX ``\pagestyle`` commands have been moved to the LaTeX template. No changes in PDF, except possibly if ``\sphinxtableofcontents``, which contained them, had been customized in :file:`conf.py`. (refs: 5455) Bugs fixed ---------- * 5418: Incorrect default path for sphinx-build -d/doctrees files * 5421: autodoc emits deprecation warning for :confval:`autodoc_default_flags` * 5422: lambda object causes PicklingError on storing environment * 5417: Sphinx fails to build with syntax error in Python 2.7.5 * 4911: add latexpdf to make.bat for non make-mode * 5436: Autodoc does not work with enum subclasses with properties/methods * 5437: autodoc: crashed on modules importing eggs * 5433: latex: ImportError: cannot import name 'DEFAULT_SETTINGS' * 5431: autodoc: ``autofunction`` emits a warning for callable objects * 5457: Fix TypeError in error message when override is prohibited * 5453: PDF builds of 'howto' documents have no page numbers * 5463: mathbase: math_role and MathDirective was disappeared in 1.8.0 * 5454: latex: Index has disappeared from PDF for Japanese documents * 5432: py domain: ``:type:`` field can't process ``:term:`` references * 5426: py domain: TypeError has been raised for class attribute ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/sphinx - Changelog: https://pyup.io/changelogs/sphinx/ - Homepage: http://sphinx-doc.org/ </details> ### Update [twine](https://pypi.org/project/twine) from **1.11.0** to **1.12.0**. <details> <summary>Changelog</summary> ### 1.12.0 ``` * :feature:`395 major` Add ``twine check`` command to check long description * :feature:`392 major` Drop support for Python 3.3 * :feature:`363` Empower ``--skip-existing`` for Artifactory repositories * 🐛`367` Avoid MD5 when Python is compiled in FIPS mode ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/twine - Changelog: https://pyup.io/changelogs/twine/ - Docs: https://twine.readthedocs.io/ </details>

This PR updates [tox](https://pypi.org/project/tox) from **3.1.2** to **3.4.0**. <details> <summary>Changelog</summary> ### 3.4.0 ``` ------------------ Bugfixes ^^^^^^^^ - add ``--exists-action w`` to default pip flags to handle better VCS dependencies (`pip documentation on this <https://pip.pypa.io/en/latest/reference/pip/exists-action-option>`_) - by :user:`gaborbernat` (`503 <https://github.com/tox-dev/tox/issues/503>`_) - instead of assuming the Python version from the base python name ask the interpreter to reveal the version for the ``ignore_basepython_conflict`` flag - by :user:`gaborbernat` (`908 <https://github.com/tox-dev/tox/issues/908>`_) - PEP-517 packaging fails with sdist already exists, fixed via ensuring the dist folder is empty before invoking the backend and `pypa/setuptools 1481 <https://github.com/pypa/setuptools/pull/1481>`_ - by :user:`gaborbernat` (`1003 <https://github.com/tox-dev/tox/issues/1003>`_) Features ^^^^^^^^ - add ``commands_pre`` and ``commands_post`` that run before and after running the ``commands`` (setup runs always, commands only if setup suceeds, teardown always - all run until the first failing command) - by :user:`gaborbernat` (`167 <https://github.com/tox-dev/tox/issues/167>`_) - ``pyproject.toml`` config support initially by just inline the tox.ini under ``tool.tox.legacy_tox_ini`` key; config source priority order is ``pyproject.toml``, ``tox.ini`` and then ``setup.cfg`` - by :user:`gaborbernat` (`814 <https://github.com/tox-dev/tox/issues/814>`_) - use the os environment variable ``TOX_SKIP_ENV`` to filter out tox environment names from the run list (set by ``envlist``) - by :user:`gaborbernat` (`824 <https://github.com/tox-dev/tox/issues/824>`_) - always set ``PIP_USER=0`` (do not install into the user site package, but inside the virtual environment created) and ``PIP_NO_DEPS=0`` (installing without dependencies can cause broken package installations) inside tox - by :user:`gaborbernat` (`838 <https://github.com/tox-dev/tox/issues/838>`_) - tox will inject some environment variables that to indicate a command is running within tox: ``TOX_WORK_DIR`` env var is set to the tox work directory, ``TOX_ENV_NAME`` is set to the current running tox environment name, ``TOX_ENV_DIR`` is set to the current tox environments working dir - by :user:`gaborbernat` (`847 <https://github.com/tox-dev/tox/issues/847>`_) - While running tox invokes various commands (such as building the package, pip installing dependencies and so on), these were printed in case they failed as Python arrays. Changed the representation to a shell command, allowing the users to quickly replicate/debug the failure on their own - by :user:`gaborbernat` (`851 <https://github.com/tox-dev/tox/issues/851>`_) - skip missing interpreters value from the config file can now be overridden via the ``--skip-missing-interpreters`` cli flag - by :user:`gaborbernat` (`903 <https://github.com/tox-dev/tox/issues/903>`_) - keep additional environments config order when listing them - by :user:`gaborbernat` (`921 <https://github.com/tox-dev/tox/issues/921>`_) - allow injecting config value inside the ini file dependent of the fact that we're connected to an interactive shell or not - by :user:`gaborbernat` (`947 <https://github.com/tox-dev/tox/issues/947>`_) - do not build sdist if skip install is specified for the envs to be run - by :user:`gaborbernat` (`974 <https://github.com/tox-dev/tox/issues/974>`_) - when verbosity level increases above two start passing through verbosity flags to pip - by :user:`gaborbernat` (`982 <https://github.com/tox-dev/tox/issues/982>`_) - when discovering the interpreter to use check if the tox host Python matches and use that if so - by :user:`gaborbernat` (`994 <https://github.com/tox-dev/tox/issues/994>`_) - ``-vv`` will print out why a virtual environment is re-created whenever this operation is triggered - by :user:`gaborbernat` (`1004 <https://github.com/tox-dev/tox/issues/1004>`_) Documentation ^^^^^^^^^^^^^ - clarify that ``python`` and ``pip`` refer to the virtual environments executable - by :user:`gaborbernat` (`305 <https://github.com/tox-dev/tox/issues/305>`_) - add Sphinx and mkdocs example of generating documentation via tox - by :user:`gaborbernat` (`374 <https://github.com/tox-dev/tox/issues/374>`_) - specify that ``setup.cfg`` tox configuration needs to be inside the ``tox:tox`` namespace - by :user:`gaborbernat` (`545 <https://github.com/tox-dev/tox/issues/545>`_) ``` ### 3.3.0 ``` ------------------ Bugfixes ^^^^^^^^ - fix ``TOX_LIMITED_SHEBANG`` when running under python3 - by :user:`asottile` (`931 <https://github.com/tox-dev/tox/issues/931>`_) Features ^^^^^^^^ - `PEP-517 <https://www.python.org/dev/peps/pep-0517/>`_ source distribution support (create a ``.package`` virtual environment to perform build operations inside) by :user:`gaborbernat` (`573 <https://github.com/tox-dev/tox/issues/573>`_) - `flit <https://flit.readthedocs.io>`_ support via implementing ``PEP-517`` by :user:`gaborbernat` (`820 <https://github.com/tox-dev/tox/issues/820>`_) - packaging now is exposed as a hook via ``tox_package(session, venv)`` - by :user:`gaborbernat` (`951 <https://github.com/tox-dev/tox/issues/951>`_) Miscellaneous ^^^^^^^^^^^^^ - Updated the VSTS build YAML to use the latest jobs and pools syntax - by :user:`davidstaheli` (`955 <https://github.com/tox-dev/tox/issues/955>`_) ``` ### 3.2.1 ``` ------------------ Bugfixes ^^^^^^^^ - ``--parallel--safe-build`` no longer cleans up its folders (``distdir``, ``distshare``, ``log``). - by :user:`gaborbernat` (`849 <https://github.com/tox-dev/tox/issues/849>`_) ``` ### 3.2.0 ``` ------------------ Features ^^^^^^^^ - Switch pip invocations to use the module ``-m pip`` instead of direct invocation. This could help avoid some of the shebang limitations. - by :user:`gaborbernat` (`935 <https://github.com/tox-dev/tox/issues/935>`_) - Ability to specify package requirements for the tox run via the ``tox.ini`` (``tox`` section under key ``requires`` - PEP-508 style): can be used to specify both plugin requirements or build dependencies. - by :user:`gaborbernat` (`783 <https://github.com/tox-dev/tox/issues/783>`_) - Allow to run multiple tox instances in parallel by providing the ``--parallel--safe-build`` flag. - by :user:`gaborbernat` (`849 <https://github.com/tox-dev/tox/issues/849>`_) ``` ### 3.1.3 ``` ------------------ Bugfixes ^^^^^^^^ - A caching issue that caused the ``develop-inst-nodeps`` action, which reinstalls the package under test, to always run has been resolved. The ``develop-inst-noop`` action, which, as the name suggests, is a no-op, will now run unless there are changes to ``setup.py`` or ``setup.cfg`` files that have not been reflected - by stephenfin (`909 <https://github.com/tox-dev/tox/issues/909>`_) Features ^^^^^^^^ - Python version testenvs are now automatically detected instead of comparing against a hard-coded list of supported versions. This enables ``py38`` and eventually ``py39`` / ``py40`` / etc. to work without requiring an upgrade to ``tox``. As such, the following public constants are now deprecated (and scheduled for removal in ``tox`` 4.0: ``CPYTHON_VERSION_TUPLES``, ``PYPY_VERSION_TUPLES``, ``OTHER_PYTHON_INTERPRETERS``, and ``DEFAULT_FACTORS`` - by :user:`asottile` (`914 <https://github.com/tox-dev/tox/issues/914>`_) Documentation ^^^^^^^^^^^^^ - Add a system overview section on the index page that explains briefly how tox works - by :user:`gaborbernat`. (`867 <https://github.com/tox-dev/tox/issues/867>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/tox - Changelog: https://pyup.io/changelogs/tox/ - Docs: https://tox.readthedocs.org/ </details>

pganssle requested changes Sep 16, 2018

View reviewed changes

gaborbernat added 3 commits September 16, 2018 22:10

build_meta sdist directory delegate to --dist-dir

d397914

Add tests exposing the issues with sdist_directory not being --dist-dir

cd0278d

add changelog for #1481

bcbc528

pganssle approved these changes Sep 16, 2018

View reviewed changes

wordsmith changelog

da5f861

jaraco merged commit 91e2d0e into pypa:master Sep 18, 2018

gaborbernat deleted the sdist-dir branch September 18, 2018 18:24

gaborbernat mentioned this pull request Sep 19, 2018

PEP-517 packaging fails with sdist already exists on version change tox-dev/tox#1003

Closed

pganssle mentioned this pull request Sep 20, 2018

build meta should honour the passed in dist folder #1476

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build_meta sdist directory delegate to --dist-dir #1481

build_meta sdist directory delegate to --dist-dir #1481

gaborbernat commented Sep 14, 2018 •

edited

Loading

pganssle commented Sep 14, 2018

pganssle commented Sep 14, 2018

gaborbernat commented Sep 14, 2018

pganssle commented Sep 14, 2018

benoit-pierre commented Sep 14, 2018

gaborbernat commented Sep 14, 2018

pganssle left a comment

pganssle commented Sep 16, 2018

gaborbernat commented Sep 16, 2018

gaborbernat commented Sep 18, 2018

pfmoore commented Sep 18, 2018

gaborbernat commented Sep 18, 2018 •

edited

Loading

pfmoore commented Sep 18, 2018

gaborbernat commented Sep 18, 2018

pfmoore commented Sep 18, 2018

jaraco commented Sep 18, 2018

jaraco commented Sep 18, 2018

gaborbernat commented Sep 18, 2018

gaborbernat commented Sep 18, 2018

build_meta sdist directory delegate to --dist-dir #1481

build_meta sdist directory delegate to --dist-dir #1481

Conversation

gaborbernat commented Sep 14, 2018 • edited Loading

pganssle commented Sep 14, 2018

pganssle commented Sep 14, 2018

gaborbernat commented Sep 14, 2018

pganssle commented Sep 14, 2018

benoit-pierre commented Sep 14, 2018

gaborbernat commented Sep 14, 2018

pganssle left a comment

Choose a reason for hiding this comment

pganssle commented Sep 16, 2018

gaborbernat commented Sep 16, 2018

gaborbernat commented Sep 18, 2018

pfmoore commented Sep 18, 2018

gaborbernat commented Sep 18, 2018 • edited Loading

pfmoore commented Sep 18, 2018

gaborbernat commented Sep 18, 2018

pfmoore commented Sep 18, 2018

jaraco commented Sep 18, 2018

jaraco commented Sep 18, 2018

gaborbernat commented Sep 18, 2018

gaborbernat commented Sep 18, 2018

gaborbernat commented Sep 14, 2018 •

edited

Loading

gaborbernat commented Sep 18, 2018 •

edited

Loading