Remove the hardcoding of python version in setup.py and SConstruct to build sage_clib and sage itself #11376

kiwifb · 2011-05-25T01:12:06Z

Currently to build sage (the spkg) four files are involved in major way:

SConstuct to build sage-clib
setup.py
module_list.py
sage/misc/cython.py

The last three to build the sage python extensions. setup.py, and SConstruct both have the version of python used hardcoded in them, module_list.py doesn't refer to the python version anywhere. cython.py mention the python version explicitly in doctest but smartly find the version used for building all by itself.

In this ticket I bring this smartness to SConstruct and setup.py. The advantage of doing this is that sage then can be build with either python-2.6 or 2.7 (and 3.x in the future) without patching. Of course it may still require patches to fix doctest and syntax in some parts - but you would be able to build sage in the first place without having to patch the build system. It has been found to be an impediment to testing in #9958.

Depend: #11373

Apply:

Depends on #11373

CC: @strogdon @nexttime

Component: build

Author: François Bissey

Reviewer: Mariah Lenox

Merged: sage-4.7.2.alpha0

Issue created by migration from https://trac.sagemath.org/ticket/11376

kiwifb · 2011-05-25T01:15:15Z

Attachment: trac_11376-build-sage_clib.patch.gz

Remove the python version hardcoding in SConstruct, remove debian cruft as well.

kiwifb · 2011-05-25T01:16:38Z

comment:1

I also remove the Debian cruft from SConstruct and setup.py at the same time.

kiwifb · 2011-05-25T01:16:38Z

Author: François Bissey

sagetrac-mariah · 2011-05-25T20:41:15Z

comment:4

patch attachment: trac_11376-build-sage_clib.patch failed to apply
to sage-4.7.rc4:

sage: hg_sage.apply("/home/mariah/trac_11376-build-sage_clib.patch")
cd "/home/mariah/sage/sage-4.7.rc4-x86_64-Linux-core2-fc/devel/sage" && hg status
cd "/home/mariah/sage/sage-4.7.rc4-x86_64-Linux-core2-fc/devel/sage" && hg status
cd "/home/mariah/sage/sage-4.7.rc4-x86_64-Linux-core2-fc/devel/sage" && hg import   "/home/mariah/trac_11376-build-sage_clib.patch"
applying /home/mariah/trac_11376-build-sage_clib.patch
patching file c_lib/SConstruct
Hunk #3 FAILED at 133
1 out of 3 hunks FAILED -- saving rejects to file c_lib/SConstruct.rej
abort: patch failed to apply
sage:

kiwifb · 2011-05-25T21:36:10Z

comment:5

Thanks for looking at this Mariah. It doesn't apply because it was based on 4.7.1.alpha1. It was based on 4.7.1.alpha1 because of #11167 I cannot produce a patch that will work equally on 4.7.rc and 4.7.1.alpha. That's unfortunate but such is life. In fact I may have to rebase attachment: trac_11376-build_setuppy.patch because of #11373 in 4.7.1.alpha2.

kiwifb · 2011-05-25T21:45:12Z

a version of this patch that works with 4.7.rc4

kiwifb · 2011-05-25T21:46:36Z

comment:6

Attachment: trac_11376-build-sage_clib-4.7.patch.gz

Just in case you still want to test it against 4.7.rc4 use this patch attachment: trac_11376-build-sage_clib-4.7.patch instead.

sagetrac-mariah · 2011-05-26T13:15:19Z

comment:7

Replying to @kiwifb:

Thanks for looking at this Mariah. It doesn't apply because it was based on 4.7.1.alpha1.

Apologies for the noise. I did not realize that 4.7.1.alpha1 was out. I have not seen any announcement of 4.7.1-alpha1 on sage-release. Is there someplace I should be looking for such announcements?

kiwifb · 2011-05-26T13:41:57Z

comment:8

Replying to @sagetrac-mariah:

Replying to @kiwifb:

Thanks for looking at this Mariah. It doesn't apply because it was based on 4.7.1.alpha1.

Apologies for the noise. I did not realize that 4.7.1.alpha1 was out. I have not seen any announcement of 4.7.1-alpha1 on sage-release. Is there someplace I should be looking for such announcements?

No apologies required. It isn't officially out but Jeroen has "prototypes" alpha0, alpha1 and alpha2 already on http://sage.math.washington.edu/home/release/.
It is a bit a pain to know that if you do something against 4.7.rc4 it will have to be put back to "needs work" once 4.7 is finally out and Jeroen officially announce the next alpha.

kiwifb · 2011-05-26T19:11:29Z

comment:9

I will add that it is perfectly acceptable to me if you want to leave this ticket for now and come back to it once a 4.7.1.alpha is released.

kiwifb · 2011-05-27T21:23:15Z

comment:10

Now that 4.7.1.alpha0 is released this ticket should be in a reviewable state.

kiwifb · 2011-05-27T22:16:16Z

Attachment: trac_11376-build_setuppy.patch.gz

Make setup.py python version smart, remove debian cruft. (made dependent on #11373)

kiwifb · 2011-05-27T22:18:43Z

comment:11

Decided to avoid problem with #11373 so make it depend on it.

jhpalmieri · 2011-06-01T19:59:37Z

comment:12

The python documentation for sys.version says "Do not extract version information out of it, rather, use version_info and the functions provided by the platform module." It looks to me as though you could use

platform.python_version()  # gives '2.6.4'
platform.python_version().rstrip('.', 1)[0]  # gives '2.6'
ver = platform.python_version_tuple()  # gives ('2', '6', '4')
ver[0] + '.' + ver[1]  # gives '2.6'

or

ver = sys.version_info
str(ver[0]) + '.' + str(ver[1])   # gives '2.6'

I'll take a look at the rest of the patches...

kiwifb · 2011-06-01T21:57:01Z

comment:13

Replying to @jhpalmieri:

The python documentation for sys.version says "Do not extract version information out of it, rather, use version_info and the functions provided by the platform module." It looks to me as though you could use
platform.python_version()  # gives '2.6.4'
platform.python_version().rstrip('.', 1)[0]  # gives '2.6'
ver = platform.python_version_tuple()  # gives ('2', '6', '4')
ver[0] + '.' + ver[1]  # gives '2.6'
or
ver = sys.version_info
str(ver[0]) + '.' + str(ver[1])   # gives '2.6'
I'll take a look at the rest of the patches...

OK I will rewrite that then. It was purely a case of stealing code from somewhere else. Looks cool in misc/cython.py that will be useful here. I should bring cython.py in line too.

jhpalmieri · 2011-06-01T22:33:12Z

comment:14

By the way, I should have provided a link: see http://docs.python.org/library/sys.html#sys.version for information about sys.version, and pointers to the other options.

kiwifb · 2011-06-02T00:37:53Z

comment:15

New version of patches all using the platform module. I also included sage/misc/cython.py to switch it from sys to platform. The cython patch in #9958 will be updated consequently.

jhpalmieri · 2011-06-02T05:06:15Z

comment:16

Can there ever be version numbers like '2.10'? Choosing the first three characters of the version string seems riskier than removing everything starting from the second dot.

sagetrac-mariah · 2011-06-14T13:10:32Z

comment:22

On skynet/eno (x86_64-Linux-core2-fc), I applied the three patches to sage-4.7.1.alpha2, did 'sage -b', then 'make testlong'. I got the following failure:

eno% ./sage -t  -long -force_lib "devel/sage/sage/misc/cython.py"
sage -t -long -force_lib "devel/sage/sage/misc/cython.py"   
**********************************************************************
File "/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/misc/cython.py", line 146:
    sage: pyx_preparse("")
Expected:
    ('\ninclude "interrupt.pxi"  # ctrl-c interrupt block support\ninclude "stdsage.pxi"  # ctrl-c interrupt block support\n\ninclude "cdefs.pxi"\n',
    ['mpfr',
    'gmp',
    'gmpxx',
    'stdc++',
    'pari',
    'm',
    'curvesntl',
    'g0nntl',
    'jcntl',
    'rankntl',
    'gsl',
    '...blas',
    ...,
    'ntl',
    'csage'],
    ['.../local/include/csage/',
    '.../local/include/',
    '.../local/include/python2.6/',
    '.../devel/sage/sage/ext/',
    '.../devel/sage/',
    '.../devel/sage/sage/gsl/'],
    'c',
    [])
Got:
    ('\ninclude "interrupt.pxi"  # ctrl-c interrupt block support\ninclude "stdsage.pxi"  # ctrl-c interrupt block support\n\ninclude "cdefs.pxi"\n', ['mpfr', 'gmp', 'gmpxx', 'stdc++', 'pari', 'm', 'curvesntl', 'g0nntl', 'jcntl', 'rankntl', 'gsl', 'gslcblas', 'atlas', 'ntl', 'csage'], ['/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/csage/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/python2.6/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//lib/python2.6/site-packages/numpy/core/include', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/ext/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/gsl/'], 'c', [])
**********************************************************************
File "/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/misc/cython.py", line 188:
    sage: inc
Expected:
    ['bar',
    '.../local/include/csage/',
    '.../local/include/',
    '.../local/include/python2.6/',
    '.../devel/sage/sage/ext/',
    '.../devel/sage/',
    '.../devel/sage/sage/gsl/']
Got:
    ['bar', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/csage/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/python2.6/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//lib/python2.6/site-packages/numpy/core/include', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/ext/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/gsl/']
**********************************************************************
1 items had failures:
   2 of  10 in __main__.example_2

kiwifb · 2011-06-14T13:20:51Z

comment:23

Hi Mariah,

I fixed this test as part of #9958 in https://github.com/sagemath/sage/files/ticket9958/trac_9958-fix-misc_cythonpy.patch.gz I thought it was appearing because of python-2.7.

If it shows up for you with python-2.6 it is strange that it didn't before the patch.

In any case if it shows up for you the above patch will have to be split and moved to this ticket. Just a few minutes.

kiwifb · 2011-06-14T13:28:32Z

Attachment: trac_11376-fix_cythonpy_doctest.patch.gz

fix the cython.py doctest for the appearance on numpy

kiwifb · 2011-06-14T13:29:38Z

comment:24

OK new patch added.

strogdon · 2011-06-14T15:46:30Z

comment:25

What about the

/local//include/ and /local//lib/

in the returned results? Could this also cause a failure? I'm seeing this also in failure of the cython.py test when the patches of #9958 are applied. And the numpy fix is already in those patches.

sagetrac-mariah · 2011-06-14T17:12:13Z

comment:26

On skynet/eno (x86_64-Linux-core2-fc), I applied all four patches to sage-4.7.1.alpha1, did 'sage -b', but I still get the error

sage -t -long -force_lib "devel/sage/sage/misc/cython.py"   
**********************************************************************
File "/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/misc/cython.py", line 146:
    sage: pyx_preparse("")
Expected:
    ('\ninclude "interrupt.pxi"  # ctrl-c interrupt block support\ninclude "stdsage.pxi"  # ctrl-c interrupt block support\n\ninclude "cdefs.pxi"\n',
    ['mpfr',
    'gmp',
    'gmpxx',
    'stdc++',
    'pari',
    'm',
    'curvesntl',
    'g0nntl',
    'jcntl',
    'rankntl',
    'gsl',
    '...blas',
    ...,
    'ntl',
    'csage'],
    ['.../local/include/csage/',
    '.../local/include/',
    '.../local/include/python2.6/',
    '.../local/lib/python2.6/site-packages/numpy/core/include',
    '.../devel/sage/sage/ext/',
    '.../devel/sage/',
    '.../devel/sage/sage/gsl/'],
    'c',
    [])
Got:
    ('\ninclude "interrupt.pxi"  # ctrl-c interrupt block support\ninclude "stdsage.pxi"  # ctrl-c interrupt block support\n\ninclude "cdefs.pxi"\n', ['mpfr', 'gmp', 'gmpxx', 'stdc++', 'pari', 'm', 'curvesntl', 'g0nntl', 'jcntl', 'rankntl', 'gsl', 'gslcblas', 'atlas', 'ntl', 'csage'], ['/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/csage/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/python2.6/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//lib/python2.6/site-packages/numpy/core/include', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/ext/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/gsl/'], 'c', [])
**********************************************************************
File "/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/misc/cython.py", line 189:
    sage: inc
Expected:
    ['bar',
    '.../local/include/csage/',
    '.../local/include/',
    '.../local/include/python2.6/',
    '.../local/lib/python2.6/site-packages/numpy/core/include',
    '.../devel/sage/sage/ext/',
    '.../devel/sage/',
    '.../devel/sage/sage/gsl/']
Got:
    ['bar', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/csage/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//include/python2.6/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/local//lib/python2.6/site-packages/numpy/core/include', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/ext/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/', '/home/mariah/sage/sage-4.7.1.alpha2-x86_64-Linux-core2-fc-review-11376/devel/sage/sage/gsl/']
**********************************************************************
1 items had failures:
   2 of  10 in __main__.example_2
***Test Failed*** 2 failures.
For whitespace errors, see the file /home/mariah/.sage//tmp/.doctest_cython.py
         [4.7 s]
 
----------------------------------------------------------------------
The following tests failed:

kiwifb · 2011-06-14T19:10:01Z

Attachment: trac_11376-msic_cythonpy-p2.patch.gz

Patch to misc/cython.py with the right number of slashes

kiwifb · 2011-06-14T19:12:52Z

comment:27

Thanks for pointing it out Steve I had missed it completely. We should have a uniform definition of SAGE_LOCAL, sometimes it has a final "/" and sometimes it doesn't. So I updated the misc_cythonpy patch to -p2. Hope we got everything now.

sagetrac-mariah · 2011-06-16T13:22:05Z

comment:28

Applied all patches to sage-4.7.1.alpha2, did 'sage -b', and 'make testlong'. All tests passed. Positive review! (finally!)

sagetrac-mariah · 2011-06-16T13:22:05Z

Reviewer: Mariah Lenox

kiwifb · 2011-06-16T23:21:52Z

comment:30

A little word of thanks, in the right bug this time (note to self don't do this again at 1:30am):

Thank you Mariah for the review. Thank you John for the suggestion to make it better. Thank you Steve for pointing the obvious.

Thank you all for making this reach this stage. Hopefully this will be in 4.7.2.alpha0.

jdemeyer · 2011-07-24T11:16:36Z

Merged: sage-4.7.2.alpha0

kiwifb added this to the sage-4.7.1 milestone May 25, 2011

kiwifb added c: build labels May 25, 2011

kiwifb assigned sagetrac-GeorgSWeber May 25, 2011

kiwifb added the s: needs review label May 25, 2011

sagetrac-mariah mannequin added s: needs work and removed s: needs review labels May 25, 2011

sagetrac-mariah mannequin added s: needs review and removed s: needs work labels May 26, 2011

This comment has been minimized.

Sign in to view

sagetrac-mariah mannequin added s: needs work and removed s: needs review labels Jun 14, 2011

This comment has been minimized.

Sign in to view

kiwifb added s: needs review and removed s: needs work labels Jun 14, 2011

sagetrac-mariah mannequin added s: needs work and removed s: needs review labels Jun 14, 2011

This comment has been minimized.

Sign in to view

kiwifb added s: needs review and removed s: needs work labels Jun 14, 2011

sagetrac-mariah mannequin added s: positive review and removed s: needs review labels Jun 16, 2011

jdemeyer modified the milestones: sage-4.7.1, sage-4.7.2 Jun 16, 2011

jdemeyer removed the s: positive review label Jul 24, 2011

jdemeyer closed this as completed Jul 24, 2011

This was referenced Jun 1, 2011

Upgrade python to 2.7.x #9958

Closed

Refcounting for Singular rings #11339

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove the hardcoding of python version in setup.py and SConstruct to build sage_clib and sage itself #11376

Remove the hardcoding of python version in setup.py and SConstruct to build sage_clib and sage itself #11376

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

sagetrac-mariah mannequin commented May 25, 2011

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

sagetrac-mariah mannequin commented May 26, 2011

kiwifb commented May 26, 2011

kiwifb commented May 26, 2011

kiwifb commented May 27, 2011

kiwifb commented May 27, 2011

This comment has been minimized.

kiwifb commented May 27, 2011

jhpalmieri commented Jun 1, 2011

kiwifb commented Jun 1, 2011

jhpalmieri commented Jun 1, 2011

kiwifb commented Jun 2, 2011

This comment has been minimized.

jhpalmieri commented Jun 2, 2011

sagetrac-mariah mannequin commented Jun 14, 2011

kiwifb commented Jun 14, 2011

kiwifb commented Jun 14, 2011

kiwifb commented Jun 14, 2011

This comment has been minimized.

strogdon commented Jun 14, 2011

sagetrac-mariah mannequin commented Jun 14, 2011

kiwifb commented Jun 14, 2011

This comment has been minimized.

kiwifb commented Jun 14, 2011

sagetrac-mariah mannequin commented Jun 16, 2011

sagetrac-mariah mannequin commented Jun 16, 2011

kiwifb commented Jun 16, 2011

jdemeyer commented Jul 24, 2011

Remove the hardcoding of python version in setup.py and SConstruct to build sage_clib and sage itself #11376

Remove the hardcoding of python version in setup.py and SConstruct to build sage_clib and sage itself #11376

Comments

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

sagetrac-mariah mannequin commented May 25, 2011

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

kiwifb commented May 25, 2011

sagetrac-mariah mannequin commented May 26, 2011

kiwifb commented May 26, 2011

kiwifb commented May 26, 2011

kiwifb commented May 27, 2011

kiwifb commented May 27, 2011

This comment has been minimized.

kiwifb commented May 27, 2011

jhpalmieri commented Jun 1, 2011

kiwifb commented Jun 1, 2011

jhpalmieri commented Jun 1, 2011

kiwifb commented Jun 2, 2011

This comment has been minimized.

jhpalmieri commented Jun 2, 2011

sagetrac-mariah mannequin commented Jun 14, 2011

kiwifb commented Jun 14, 2011

kiwifb commented Jun 14, 2011

kiwifb commented Jun 14, 2011

This comment has been minimized.

strogdon commented Jun 14, 2011

sagetrac-mariah mannequin commented Jun 14, 2011

kiwifb commented Jun 14, 2011

This comment has been minimized.

kiwifb commented Jun 14, 2011

sagetrac-mariah mannequin commented Jun 16, 2011

sagetrac-mariah mannequin commented Jun 16, 2011

kiwifb commented Jun 16, 2011

jdemeyer commented Jul 24, 2011