Problem building flint

[jachymb]

I am getting the following error (installing via pip install . but similar error appears for the other methods suggested here https://fredrikj.net/python-flint/setup.html )

$ pip install .
Defaulting to user installation because normal site-packages is not writeable
Processing ./python-flint
Using legacy 'setup.py install' for python-flint, since package 'wheel' is not installed.
Installing collected packages: python-flint
    Running setup.py install for python-flint ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-3jim2rol/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-3jim2rol/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2kuawotm/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/jachym/.local/include/python3.9/python-flint                                                                                      
         cwd: /tmp/pip-req-build-3jim2rol/                                                                                                                   
    Complete output (87 lines):                                                                                                                              
    running install                                                                                                                                          
    running build                                                                                                                                            
    running build_ext                                                                                                                                        
    cythoning src/pyflint.pyx to src/pyflint.c                                                                                                               
    /usr/lib/python3.9/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-req-build-3jim2rol/src/flint.pxd                                                                                   
      tree = Parsing.p_module(s, pxd, full_module_name)                                                                                                      
    warning: src/flint.pxd:26:4: 'mp_limb_t' redeclared                                                                                                      
    warning: src/pyflint.pyx:13:4: 'PyObject' redeclared                                                                                                     
    warning: src/pyflint.pyx:14:4: 'PyTypeObject' redeclared                                                                                                 
    warning: src/pyflint.pyx:15:4: 'Py_ssize_t' redeclared                                                                                                   
    warning: src/pyflint.pyx:70:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables                   
    warning: src/pyflint.pyx:71:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables                   
    warning: src/pyflint.pyx:72:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables                   
    warning: src/pyflint.pyx:73:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables                   
    warning: src/pyflint.pyx:74:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables                   
    warning: src/pyflint.pyx:75:10: cpdef variables will not be supported in Cython 3; currently they are no different from cdef variables                   
    Traceback (most recent call last):                                                                                                                       
      File "<string>", line 1, in <module>                                                                                                                   
      File "/tmp/pip-req-build-3jim2rol/setup.py", line 33, in <module>                                                                                      
        setup(                                                                                                                                               
      File "/usr/lib/python3.9/distutils/core.py", line 148, in setup                                                                                        
        dist.run_commands()                                                                                                                                  
      File "/usr/lib/python3.9/distutils/dist.py", line 966, in run_commands                                                                                 
        self.run_command(cmd)                                                                                                                                
      File "/usr/lib/python3.9/distutils/dist.py", line 985, in run_command                                                                                  
        cmd_obj.run()                                                                                                                                        
      File "/usr/lib/python3.9/site-packages/setuptools/command/install.py", line 61, in run                                                                 
        return orig.install.run(self)                                                                                                                        
      File "/usr/lib/python3.9/distutils/command/install.py", line 546, in run                                                                               
        self.run_command('build')                                                                                                                            
      File "/usr/lib/python3.9/distutils/cmd.py", line 313, in run_command                                                                                   
        self.distribution.run_command(command)                                                                                                               
      File "/usr/lib/python3.9/distutils/dist.py", line 985, in run_command                                                                                  
        cmd_obj.run()                                                                                                                                        
      File "/usr/lib/python3.9/distutils/command/build.py", line 135, in run                                                                                 
        self.run_command(cmd_name)                                                                                                                           
      File "/usr/lib/python3.9/distutils/cmd.py", line 313, in run_command                                                                                   
        self.distribution.run_command(command)                                                                                                               
      File "/usr/lib/python3.9/distutils/dist.py", line 985, in run_command                                                                                  
        cmd_obj.run()                                                                                                                                        
      File "/usr/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run                                                            
        _build_ext.build_ext.run(self)                                                                                                                       
      File "/usr/lib/python3.9/distutils/command/build_ext.py", line 340, in run                                                                             
        self.build_extensions()                                                                                                                              
      File "/usr/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 193, in build_extensions                                               
        ext.sources = self.cython_sources(ext.sources, ext)                                                                                                  
      File "/usr/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 346, in cython_sources                                                 
        result = cython_compile(source, options=options,                                                                                                     
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Main.py", line 778, in compile                                                                  
        return compile_single(source, options, full_module_name)                                                                                             
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Main.py", line 727, in compile_single                                                           
        return run_pipeline(source, options, full_module_name)                                                                                               
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Main.py", line 515, in run_pipeline                                                             
        err, enddata = Pipeline.run_pipeline(pipeline, source)                                                                                               
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Pipeline.py", line 355, in run_pipeline                                                         
        data = run(phase, data)                                                                                                                              
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Pipeline.py", line 335, in run                                                                  
        return phase(data)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Pipeline.py", line 52, in generate_pyx_code_stage
        module_node.process_implementation(options, result)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/ModuleNode.py", line 143, in process_implementation
        self.generate_c_code(env, options, result)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/ModuleNode.py", line 385, in generate_c_code
        self.body.generate_function_definitions(env, code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 442, in generate_function_definitions
        stat.generate_function_definitions(env, code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 442, in generate_function_definitions
        stat.generate_function_definitions(env, code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 1986, in generate_function_definitions
        self.generate_function_body(env, code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 1748, in generate_function_body
        self.body.generate_execution_code(code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 448, in generate_execution_code
        stat.generate_execution_code(code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 5190, in generate_execution_code
        self.generate_rhs_evaluation_code(code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/Nodes.py", line 5477, in generate_rhs_evaluation_code
        self.rhs.generate_evaluation_code(code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/ExprNodes.py", line 773, in generate_evaluation_code
        self.generate_result_code(code)
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/ExprNodes.py", line 13341, in generate_result_code
        code.putln(self.type.from_py_call_code(
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/PyrexTypes.py", line 511, in from_py_call_code
        return self.typedef_base_type.from_py_call_code(
      File "/usr/lib/python3.9/site-packages/Cython/Compiler/PyrexTypes.py", line 2486, in from_py_call_code
        assert not error_condition, '%s: %s' % (error_pos, error_condition)
    AssertionError: (<StringSourceDescriptor:(tree fragment)>, 12, 34): PyErr_Occurred()
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-3jim2rol/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-3jim2rol/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-2kuawotm/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/jachym/.local/include/python3.9/python-flint Check the logs for full command output.

[jachymb]

The cause of the problem is apparently something in newer versions of cython. The error does not seem to occur with cython <= 0.27.

[oscarbenjamin]

I was able to get this to work with cython == 0.27.3. I had problems with earlier version in python 3.8 and later versions hit the error shown above.

[fredrik-johansson]

I am also hitting this error now and can no longer build python-flint :-(

This is clearly a cython bug. It may well be that python-flint is broken somehow, but cython should give some reasonable description of the problem rather than crashing with an internal assertion error.

[oscarbenjamin]

Does it not work with Cython 0.27.3?

This is a Cython bug and I thought I had seen an open issue for Cython with a much simpler reproducer although the issue isn't linked here. From memory it was reported but nothing had been done to debug it. Probably bisecting to find the first problematic Cython commit would be helpful.

[fredrik-johansson]

Yeah, possibly. I have Cython 0.29.23 on my system after upgrading my Linux distribution.

[oscarbenjamin]

The build scripts I added can build a wheel with all versions pinned so should still work with the same versions:
https://github.com/fredrik-johansson/python-flint/blob/1a68cffc3b8454f57f5c5ffdd94df3f11b081468/bin/build_wheel.sh#L15-L16

[oscarbenjamin]

(Maybe the numpy version should have been pinned but I didn't have any issues with that)

[oscarbenjamin]

I just checked that the wheel build script still works on OSX. Just run

$ bin/build_dependencies_unix.sh 
$ bin/build_wheel.sh 

That used to work on both OSX and Linux.

At least for me the wheel needs to be renamed (change the OSX version) because pip/setuptools does not recognise recent OSX versions but otherwise it works fine.

It should also be possible to build the dependencies with --gmp mpir but that currently fails because of a wrong YASM version. That can be fixed with

diff --git a/bin/build_variables.sh b/bin/build_variables.sh
index 027532c..9042685 100644
--- a/bin/build_variables.sh
+++ b/bin/build_variables.sh
@@ -14,7 +14,7 @@ PREFIX=$(pwd)/.local
 mkdir -p $PREFIX
 
 GMPVER=6.2.1
-YASMVER=1.3.1
+YASMVER=1.3.0
 MPIRVER=3.0.0
 MPFRVER=4.1.0
 FLINTVER=2.7.1

I'm not sure how that mistake crept in because I can't see any evidence that there ever was a 1.3.1 release of YASM...

After fixing the YASM version building MPIR on OSX fails with a configure error.

...
checking compiler cc -O2 -fomit-frame-pointer ... no, long long reliability test 2
configure: error: could not find a working compiler, see config.log for details

The MPIR config.log has

...
Test compile: long long reliability test 2
configure:5847: cc -O2 -fomit-frame-pointer  conftest.c >&5
conftest.c:6:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
f(int u){int i;long long x;x=u?~0:0;if(x)for(i=0;i<9;i++);x&=g();if(x)g();}
^
conftest.c:6:62: error: implicit declaration of function 'g' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
f(int u){int i;long long x;x=u?~0:0;if(x)for(i=0;i<9;i++);x&=g();if(x)g();}
                                                             ^
conftest.c:6:75: warning: non-void function does not return a value [-Wreturn-type]
f(int u){int i;long long x;x=u?~0:0;if(x)for(i=0;i<9;i++);x&=g();if(x)g();}
                                                                          ^
conftest.c:7:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
g(){}
^
conftest.c:7:5: warning: non-void function does not return a value [-Wreturn-type]
g(){}
    ^
4 warnings and 1 error generated.
configure:5850: $? = 1
failed program was:
/* The following provokes an internal compiler error on powerpc-apple-darwin.
   Extracted from mpz/cfdiv_q_2exp.c.  Causes Apple's gcc 3.3 build 1640 and
   1666 to get an ICE with -O1 -mpowerpc64.  */

#ifdef __GNUC__
f(int u){int i;long long x;x=u?~0:0;if(x)for(i=0;i<9;i++);x&=g();if(x)g();}
g(){}
#else
int dummy;
#endif

int main () { return 0; }
configure:6195: result: no, long long reliability test 2
configure:6355: error: could not find a working compiler, see config.log for details

The configure checks in a few of these libraries are quite flakey under clang I think because it defaults to C99 rather than C89.

[fredrik-johansson]

There is a possible workaround in b3eb354.

[oscarbenjamin]

The orginal issue is "fixed" now. Probably it was/is a cython bug but in any case with current python-flint and current cython you should not see this problem.

To build with pip you should first install cython>=3 and then pass --no-build-isolation (see instructions in the README). For OSX and Windows pip install python-flint will install the binaries from PyPI (see #51 for the Linux wheels).