No longer hard-code pythons #914

asottile · 2018-07-17T02:42:02Z

WIP -- I'll fill out changelog and such if it works and if it seems like a good idea. It always struck me as odd that tox hardcoded this list of interpreters and that I'd be forced to upgrade my tox installation to use new things. This patch adds support for python2.0 - python9.9, hopefully leaving us some headroom.

Here's hoping python3.10 isn't coming out soon, eh?

If this is good, the lists of interpreters can be deprecated and eventually removed I would imagine

asottile · 2018-07-17T02:43:13Z

for instance:

$ tox -e py28 --notest
GLOB sdist-make: /tmp/tox/setup.py
py28 create: /tmp/tox/.tox/py28
SKIPPED: InterpreterNotFound: python2.8
___________________________________ summary ____________________________________
SKIPPED:  py28: InterpreterNotFound: python2.8
  congratulations :)
$ tox -e py99 --notest
GLOB sdist-make: /tmp/tox/setup.py
py99 create: /tmp/tox/.tox/py99
SKIPPED: InterpreterNotFound: python9.9
___________________________________ summary ____________________________________
SKIPPED:  py99: InterpreterNotFound: python9.9
  congratulations :)
$ tox -e py6 --notest
GLOB sdist-make: /tmp/tox/setup.py
py6 create: /tmp/tox/.tox/py6
SKIPPED: InterpreterNotFound: python6
___________________________________ summary ____________________________________
SKIPPED:  py6: InterpreterNotFound: python6
  congratulations :)

obestwalter · 2018-07-17T10:21:33Z

Good idea :)

gaborbernat · 2018-07-17T11:01:20Z

Why not add support for (\d).(\d){2,}?

gaborbernat · 2018-07-17T11:01:36Z

Note the VSTS CI failed with valid failures.

asottile · 2018-07-17T14:15:23Z

Why not add support for (\d).(\d){2,}?

because it doesn't already exist -- wanted to replicate existing behaviour as much as possible

The failure as far as I can tell isn't actually an issue since python falls to sys.executable anyway (just not at that portion in the code) but I'll fix before shipping

asottile · 2018-07-17T14:17:21Z

@gaborbernat did you force push master? my history no longer aligns

gaborbernat · 2018-07-17T14:19:25Z

Yes sadly by mistake, and had to rebuild master 👎 if you want to force push back and cherry pick the one new merge you're free 👍 made the master protected.

asottile · 2018-07-17T14:21:24Z

ok, I'm going to reset to e80f69e and then re-apply cc3d3e6

asottile · 2018-07-17T14:25:46Z

oh right, need to unprotect to force push myself. I've put the old master I'm about to force push over at _old-master (we can delete this branch once I'm done and we're happy)

asottile · 2018-07-17T14:27:37Z

@gaborbernat can you re setup the branch protection (if master looks good) -- I've forgotten which checks were required 🤦‍♂️

gaborbernat · 2018-07-17T15:27:18Z

sure 🆗

gaborbernat · 2018-07-17T15:47:27Z

Done, your tests still fail at:

2018-07-17T14:35:36.9813333Z     def test_py_venv(self, newconfig):
2018-07-17T14:35:36.9829608Z         config = newconfig(["-epy"], "")
2018-07-17T14:35:36.9846066Z         env = config.envconfigs["py"]
2018-07-17T14:35:36.9861785Z >       assert str(env.basepython) == sys.executable
2018-07-17T14:35:36.9877452Z E       AssertionError: assert 'python' == '/opt/vsts/work/1/s/.tox/py/bin/python'
2018-07-17T14:35:36.9894093Z E         - python
2018-07-17T14:35:36.9909134Z E         + /opt/vsts/work/1/s/.tox/py/bin/python
2018-07-17T14:35:36.9917827Z 
2018-07-17T14:35:36.9939780Z config     = <tox.config.Config object at 0x7f84845f56d0>
2018-07-17T14:35:36.9955010Z env        = <tox.config.TestenvConfig instance at 0x7f84842a3d88>
2018-07-17T14:35:36.9970433Z newconfig  = <function create_new_config_file_ at 0x7f84841be578>

asottile · 2018-07-17T15:57:33Z

yeah haven't "fixed" that yet -- just rebased :)

codecov · 2018-07-17T17:42:31Z

Codecov Report

Merging #914 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #914      +/-   ##
==========================================
+ Coverage   92.61%   92.63%   +0.01%     
==========================================
  Files          12       12              
  Lines        2330     2335       +5     
  Branches      409      409              
==========================================
+ Hits         2158     2163       +5     
  Misses        109      109              
  Partials       63       63

Impacted Files	Coverage Δ
src/tox/config.py	`95.91% <100%> (+0.01%)`	⬆️
src/tox/constants.py	`100% <100%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66bbfef...f837b1a. Read the comment docs.

gaborbernat

Is this ready then @asottile? In general, I agree with the approach although I think we might as well make the minor take two version numbers, not just one. I know 3.10 is not out yet, but neither is 3.9 and we support that via this construction. Will need a news fragment, otherwise looks great.

gaborbernat · 2018-07-18T09:04:54Z

src/tox/config.py

-                default = tox.PYTHON.DEFAULT_FACTORS[factor]
+            match = tox.PYTHON.PY_FACTORS_RE.match(factor)
+            if match:
+                default = tox.PYTHON.PY_FACTORS_MAP[match.group(1)] + ".".join(


would prefer a format here 🤔

asottile · 2018-07-18T15:39:05Z

According to http://www.curiousefficiency.org/posts/2014/08/python-4000.html 3.10 will never come -- so I'm going to leave this at two digits for now

Curious Efficiency
Why Python 4.0 won't be like Python 3.0
Newcomers to python-ideas occasionally make reference to the idea of "Python 4000" when proposing backwards incompatible changes that don't offer a clear migration path from currently legal Python 3 c

obestwalter · 2018-07-18T19:39:09Z

My current expectation is that Python 4.0 will merely be "the release that comes after Python 3.9".

Sounds simple enough :)

gaborbernat · 2018-07-19T08:50:36Z

Taken from there what you linked:

Update: After this post was originally written back in 2014, subsequent discussions on the core python-dev mailing list led to the conclusion that the release after 3.9 will probably just be 3.10. However, a 4.0 will presumably still happen some day, and the premise of this article is expected to hold for that release: it will be held to the same backwards compatibility obligations as a Python 3.X to 3.X+1 update.

So there highly likely will be a 3.10.

obestwalter · 2018-07-19T09:07:09Z

oh, ok then maybe support it right away after all, then we can't forget it in 2023 ...

gaborbernat

I'm fine to merge this at it is. However just pointing out that py40 probably will come after py310.

asottile · 2018-07-19T14:26:04Z

@gaborbernat is the "this branch is out of date" thing new? Really don't like it, it is very tedious.

gaborbernat · 2018-07-19T14:42:33Z

Fixed.

anlutro · 2018-08-03T07:11:05Z

Could we get a pypi release with this fix included? 3.1.2 is affected by the bug. I was super confused when -e py33 was working despite python3.3 not being installed.

gaborbernat · 2018-08-03T07:25:55Z

@anlutro are you sure you're referring to this? This PR allows specifying python 3.9 and so on. Has not much to do with -e py33.

anlutro · 2018-08-03T07:40:56Z

Yes, I'm sure. If I revert the commit of this PR, tox -e py33 creates a virtualenv using python3.6.

gaborbernat · 2018-08-03T08:09:51Z

Can you provide more details in an issue, please? Thanks!

anlutro · 2018-08-03T08:22:15Z

Sure, if you insist, but the problem is already solved in master by this PR.

anlutro · 2018-08-03T08:29:21Z

Opened issue here: #932

This PR updates [tox](https://pypi.org/project/tox) from **3.1.2** to **3.4.0**. <details> <summary>Changelog</summary> ### 3.4.0 ``` ------------------ Bugfixes ^^^^^^^^ - add ``--exists-action w`` to default pip flags to handle better VCS dependencies (`pip documentation on this <https://pip.pypa.io/en/latest/reference/pip/exists-action-option>`_) - by :user:`gaborbernat` (`503 <https://github.com/tox-dev/tox/issues/503>`_) - instead of assuming the Python version from the base python name ask the interpreter to reveal the version for the ``ignore_basepython_conflict`` flag - by :user:`gaborbernat` (`908 <https://github.com/tox-dev/tox/issues/908>`_) - PEP-517 packaging fails with sdist already exists, fixed via ensuring the dist folder is empty before invoking the backend and `pypa/setuptools 1481 <https://github.com/pypa/setuptools/pull/1481>`_ - by :user:`gaborbernat` (`1003 <https://github.com/tox-dev/tox/issues/1003>`_) Features ^^^^^^^^ - add ``commands_pre`` and ``commands_post`` that run before and after running the ``commands`` (setup runs always, commands only if setup suceeds, teardown always - all run until the first failing command) - by :user:`gaborbernat` (`167 <https://github.com/tox-dev/tox/issues/167>`_) - ``pyproject.toml`` config support initially by just inline the tox.ini under ``tool.tox.legacy_tox_ini`` key; config source priority order is ``pyproject.toml``, ``tox.ini`` and then ``setup.cfg`` - by :user:`gaborbernat` (`814 <https://github.com/tox-dev/tox/issues/814>`_) - use the os environment variable ``TOX_SKIP_ENV`` to filter out tox environment names from the run list (set by ``envlist``) - by :user:`gaborbernat` (`824 <https://github.com/tox-dev/tox/issues/824>`_) - always set ``PIP_USER=0`` (do not install into the user site package, but inside the virtual environment created) and ``PIP_NO_DEPS=0`` (installing without dependencies can cause broken package installations) inside tox - by :user:`gaborbernat` (`838 <https://github.com/tox-dev/tox/issues/838>`_) - tox will inject some environment variables that to indicate a command is running within tox: ``TOX_WORK_DIR`` env var is set to the tox work directory, ``TOX_ENV_NAME`` is set to the current running tox environment name, ``TOX_ENV_DIR`` is set to the current tox environments working dir - by :user:`gaborbernat` (`847 <https://github.com/tox-dev/tox/issues/847>`_) - While running tox invokes various commands (such as building the package, pip installing dependencies and so on), these were printed in case they failed as Python arrays. Changed the representation to a shell command, allowing the users to quickly replicate/debug the failure on their own - by :user:`gaborbernat` (`851 <https://github.com/tox-dev/tox/issues/851>`_) - skip missing interpreters value from the config file can now be overridden via the ``--skip-missing-interpreters`` cli flag - by :user:`gaborbernat` (`903 <https://github.com/tox-dev/tox/issues/903>`_) - keep additional environments config order when listing them - by :user:`gaborbernat` (`921 <https://github.com/tox-dev/tox/issues/921>`_) - allow injecting config value inside the ini file dependent of the fact that we're connected to an interactive shell or not - by :user:`gaborbernat` (`947 <https://github.com/tox-dev/tox/issues/947>`_) - do not build sdist if skip install is specified for the envs to be run - by :user:`gaborbernat` (`974 <https://github.com/tox-dev/tox/issues/974>`_) - when verbosity level increases above two start passing through verbosity flags to pip - by :user:`gaborbernat` (`982 <https://github.com/tox-dev/tox/issues/982>`_) - when discovering the interpreter to use check if the tox host Python matches and use that if so - by :user:`gaborbernat` (`994 <https://github.com/tox-dev/tox/issues/994>`_) - ``-vv`` will print out why a virtual environment is re-created whenever this operation is triggered - by :user:`gaborbernat` (`1004 <https://github.com/tox-dev/tox/issues/1004>`_) Documentation ^^^^^^^^^^^^^ - clarify that ``python`` and ``pip`` refer to the virtual environments executable - by :user:`gaborbernat` (`305 <https://github.com/tox-dev/tox/issues/305>`_) - add Sphinx and mkdocs example of generating documentation via tox - by :user:`gaborbernat` (`374 <https://github.com/tox-dev/tox/issues/374>`_) - specify that ``setup.cfg`` tox configuration needs to be inside the ``tox:tox`` namespace - by :user:`gaborbernat` (`545 <https://github.com/tox-dev/tox/issues/545>`_) ``` ### 3.3.0 ``` ------------------ Bugfixes ^^^^^^^^ - fix ``TOX_LIMITED_SHEBANG`` when running under python3 - by :user:`asottile` (`931 <https://github.com/tox-dev/tox/issues/931>`_) Features ^^^^^^^^ - `PEP-517 <https://www.python.org/dev/peps/pep-0517/>`_ source distribution support (create a ``.package`` virtual environment to perform build operations inside) by :user:`gaborbernat` (`573 <https://github.com/tox-dev/tox/issues/573>`_) - `flit <https://flit.readthedocs.io>`_ support via implementing ``PEP-517`` by :user:`gaborbernat` (`820 <https://github.com/tox-dev/tox/issues/820>`_) - packaging now is exposed as a hook via ``tox_package(session, venv)`` - by :user:`gaborbernat` (`951 <https://github.com/tox-dev/tox/issues/951>`_) Miscellaneous ^^^^^^^^^^^^^ - Updated the VSTS build YAML to use the latest jobs and pools syntax - by :user:`davidstaheli` (`955 <https://github.com/tox-dev/tox/issues/955>`_) ``` ### 3.2.1 ``` ------------------ Bugfixes ^^^^^^^^ - ``--parallel--safe-build`` no longer cleans up its folders (``distdir``, ``distshare``, ``log``). - by :user:`gaborbernat` (`849 <https://github.com/tox-dev/tox/issues/849>`_) ``` ### 3.2.0 ``` ------------------ Features ^^^^^^^^ - Switch pip invocations to use the module ``-m pip`` instead of direct invocation. This could help avoid some of the shebang limitations. - by :user:`gaborbernat` (`935 <https://github.com/tox-dev/tox/issues/935>`_) - Ability to specify package requirements for the tox run via the ``tox.ini`` (``tox`` section under key ``requires`` - PEP-508 style): can be used to specify both plugin requirements or build dependencies. - by :user:`gaborbernat` (`783 <https://github.com/tox-dev/tox/issues/783>`_) - Allow to run multiple tox instances in parallel by providing the ``--parallel--safe-build`` flag. - by :user:`gaborbernat` (`849 <https://github.com/tox-dev/tox/issues/849>`_) ``` ### 3.1.3 ``` ------------------ Bugfixes ^^^^^^^^ - A caching issue that caused the ``develop-inst-nodeps`` action, which reinstalls the package under test, to always run has been resolved. The ``develop-inst-noop`` action, which, as the name suggests, is a no-op, will now run unless there are changes to ``setup.py`` or ``setup.cfg`` files that have not been reflected - by stephenfin (`909 <https://github.com/tox-dev/tox/issues/909>`_) Features ^^^^^^^^ - Python version testenvs are now automatically detected instead of comparing against a hard-coded list of supported versions. This enables ``py38`` and eventually ``py39`` / ``py40`` / etc. to work without requiring an upgrade to ``tox``. As such, the following public constants are now deprecated (and scheduled for removal in ``tox`` 4.0: ``CPYTHON_VERSION_TUPLES``, ``PYPY_VERSION_TUPLES``, ``OTHER_PYTHON_INTERPRETERS``, and ``DEFAULT_FACTORS`` - by :user:`asottile` (`914 <https://github.com/tox-dev/tox/issues/914>`_) Documentation ^^^^^^^^^^^^^ - Add a system overview section on the index page that explains briefly how tox works - by :user:`gaborbernat`. (`867 <https://github.com/tox-dev/tox/issues/867>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/tox - Changelog: https://pyup.io/changelogs/tox/ - Docs: https://tox.readthedocs.org/ </details>

Mainly to get a version that includes tox-dev/tox#914

gaborbernat force-pushed the master branch from e80f69e to bb4a94c Compare July 17, 2018 11:31

asottile force-pushed the master branch from cc3d3e6 to 95f5476 Compare July 17, 2018 14:26

asottile mentioned this pull request Jul 17, 2018

Tox envlist selects incorrect python versions #882

Closed

gaborbernat reviewed Jul 18, 2018

View reviewed changes

gaborbernat added feature:new something does not exist yet, but should area:configuration level:medium rought estimate that this might be neither easy nor hard to implement labels Jul 18, 2018

gaborbernat added this to the 3.2 milestone Jul 18, 2018

No longer hard-code pythons

f837b1a

gaborbernat approved these changes Jul 19, 2018

View reviewed changes

gaborbernat merged commit 95b9e1c into tox-dev:master Jul 19, 2018

asottile deleted the unhardcode_pythons branch July 19, 2018 14:43

asottile mentioned this pull request Jul 27, 2018

bug bad python version #927

Closed

anlutro mentioned this pull request Aug 3, 2018

Environments specifying older versions of Python are created with newer versions of python #932

Closed

asottile mentioned this pull request Mar 17, 2019

Detect py38 as python3.8 #913

Merged

tomasaschan added a commit to tomasaschan/chartify that referenced this pull request Apr 22, 2020

Use latest tox

d34a854

Mainly to get a version that includes tox-dev/tox#914

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

No longer hard-code pythons #914

No longer hard-code pythons #914

asottile commented Jul 17, 2018

asottile commented Jul 17, 2018

obestwalter commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

asottile commented Jul 17, 2018

asottile commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

asottile commented Jul 17, 2018

asottile commented Jul 17, 2018

asottile commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

asottile commented Jul 17, 2018

codecov bot commented Jul 17, 2018 •

edited

Loading

gaborbernat left a comment

gaborbernat Jul 18, 2018

asottile commented Jul 18, 2018 •

edited by unfurl-links bot

Loading

obestwalter commented Jul 18, 2018

gaborbernat commented Jul 19, 2018 •

edited

Loading

obestwalter commented Jul 19, 2018

gaborbernat left a comment

asottile commented Jul 19, 2018

gaborbernat commented Jul 19, 2018

anlutro commented Aug 3, 2018

gaborbernat commented Aug 3, 2018

anlutro commented Aug 3, 2018

gaborbernat commented Aug 3, 2018

anlutro commented Aug 3, 2018

anlutro commented Aug 3, 2018

No longer hard-code pythons #914

No longer hard-code pythons #914

Conversation

asottile commented Jul 17, 2018

asottile commented Jul 17, 2018

obestwalter commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

asottile commented Jul 17, 2018

asottile commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

asottile commented Jul 17, 2018

asottile commented Jul 17, 2018

asottile commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

gaborbernat commented Jul 17, 2018

asottile commented Jul 17, 2018

codecov bot commented Jul 17, 2018 • edited Loading

Codecov Report

gaborbernat left a comment

Choose a reason for hiding this comment

gaborbernat Jul 18, 2018

Choose a reason for hiding this comment

asottile commented Jul 18, 2018 • edited by unfurl-links bot Loading

obestwalter commented Jul 18, 2018

gaborbernat commented Jul 19, 2018 • edited Loading

obestwalter commented Jul 19, 2018

gaborbernat left a comment

Choose a reason for hiding this comment

asottile commented Jul 19, 2018

gaborbernat commented Jul 19, 2018

anlutro commented Aug 3, 2018

gaborbernat commented Aug 3, 2018

anlutro commented Aug 3, 2018

gaborbernat commented Aug 3, 2018

anlutro commented Aug 3, 2018

anlutro commented Aug 3, 2018

codecov bot commented Jul 17, 2018 •

edited

Loading

asottile commented Jul 18, 2018 •

edited by unfurl-links bot

Loading

gaborbernat commented Jul 19, 2018 •

edited

Loading