Perl packages for polymake

[mkoeppe]

To simplify installation of polymake, we add some required CPAN packages.

Including recursive dependencies, these are 103 packages, a total of 7.5 MB of source archives, which we pack up as the source tar ball of the new spkg perl_cpan_polymake_prereq.
The source tar ball (attached) also contains a copy of the self-contained cpanm Perl script, which is used for installing the packages.
If SAGE_CHECK=no, only about 35 packages are installed. The rest of the dependencies are for testing.

Using an spkg-configure.m4, it is downloaded and installed only if the Perl packages are not already present on the system.

The spkg-install installs the packages via the package dependencies mechanism. If some of the packages are already installed in some version on the system, we do not install a newer version.

To try out:

   wget -P upstream https://github.com/sagemath/sage-prod/files/10659280/perl_cpan_polymake_prereq-2019-05-19.tar.gz

See also the sage-devel post about compilation: https://groups.google.com/forum/#!topic/sage-devel/ukJgMweHszU

Depends on #29582

CC: @dimpase @jplab [email protected] [email protected] [email protected]

Component: packages: optional

Author: Matthias Koeppe

Branch/Commit: u/mkoeppe/perl_packages_for_polymake @ 225dea9

Issue created by migration from https://trac.sagemath.org/ticket/27763

[dimpase]

comment:2

I'd suggest to have appropriate spkg-configure.m4, to avoid installing extra...

By the way, could you provide spkg-configure.m4 for perl_term_readline_gnu, and other already present as packages polymake deps (perl or not)?

[mkoeppe]

comment:3

Do you have a suggestion how to install these CPAN packages if they are not available?
There seems to be a large number of dependencies that each of the listed packages pulls in -- but probably many are "standard" on any system that has perl installed.

[dimpase]

comment:4

I don't know much about perl. Does it have anything analogous to Python's pip, which allows direct installation from the net (from CPAN, I guess) ?

[mkoeppe]

comment:5

Yes, the cpan program does the equivalent of pip.

[dimpase]

comment:6

I tried something like perl -MCPAN -e 'install XML::Writer' but it seems to want to install stuff into /usr/local rather than $SAGE_LOCAL.
Do you know how to set all this up properly?

[mkoeppe]

comment:7

I have only used cpan -i, which asks whether to install system wide or in the user's home directory.
Here's some info: https://stackoverflow.com/questions/540640/how-can-i-install-a-cpan-module-into-a-local-directory

I used something like this for perl_term_readline_gnu, but this is not going through cpan.

[dimpase]

comment:8

I don't know, it seems one needs to setup some local::lib magic, otherwise I keep getting errors trying to to install into /usr/local.

local::lib is a Perl module that apparently allows one to use non-standard directories for module installation, it might not be in by default (e.g. it's not in Gentoo, one needs to install it).

Anyhow - for checking in autoconf which perl modules are installed, one can use AX_PROG_PERL_MODULES

[dimpase]

comment:9

I also had to install File::Slurp perl module. Fortunately on Gentoo there seems to be an (almost?) 1-to-1 correspondence between perl packages named X::Y and Gentoo packages named X-Y.

I also see

File "src/sage/interfaces/polymake.py", line 884, in sage.interfaces.polymake.Polymake._eval_line
Failed example:
    c                                 # optional - polymake
Expected:
    cube of dimension 15
Got:
    1
    print ref(@SAGE590);
    print reftype(@SAGE590);
**********************************************************************
File "src/sage/interfaces/polymake.py", line 886, in sage.interfaces.polymake.Polymake._eval_line
Failed example:
    c.N_VERTICES                      # optional - polymake
Expected:
    32768
Got:
    Member function 'N_VERTICES' of  object

[mkoeppe]

comment:10

Replying to @dimpase:

I also had to install File::Slurp perl module.

Yes, this is already mentioned in polymake/SPKG.txt

Fortunately on Gentoo there seems to be an (almost?) 1-to-1 correspondence between perl packages named X::Y and Gentoo packages named X-Y.

I also see

File "src/sage/interfaces/polymake.py", line 884, in sage.interfaces.polymake.Polymake._eval_line
Failed example:
    c                                 # optional - polymake
Expected:
    cube of dimension 15
Got:
    1
    print ref(@SAGE590);
    print reftype(@SAGE590);
**********************************************************************
File "src/sage/interfaces/polymake.py", line 886, in sage.interfaces.polymake.Polymake._eval_line
Failed example:
    c.N_VERTICES                      # optional - polymake
Expected:
    32768
Got:
    Member function 'N_VERTICES' of  object

With which version of polymake is this? The 3.4 upgrade ticket has fixes for something like this.

[dimpase]

comment:11

If I do

--- a/src/sage/interfaces/polymake.py
+++ b/src/sage/interfaces/polymake.py
@@ -874,10 +874,6 @@ class Polymake(ExtraTabCompletion, Expect):
         sometimes it hangs, and therefore we remove it from the tests, for now::
 
             sage: c = polymake.cube(15)             # optional - polymake
-            sage: polymake.eval('print {}->F_VECTOR;'.format(c.name()), timeout=1) # optional - polymake # not tested
-            Traceback (most recent call last):
-            ...
-            RuntimeError: Polymake fails to respond timely
 
         We verify that after the timeout, polymake is still able to give answers::
 

then the tests pass, so the timeout thing seems to be doing more harm than good.

[dimpase]

comment:12

I am testing this branch, so 3.4.

[dimpase]

comment:13

oops, sorry, wrong ticket. This is meant to be about polymake update to 3.4

[mkoeppe]

comment:14

yes, #24905.

[dimpase]

comment:15

Please see #27795 for using system's perm package Term::ReadLine::Gnu

[mkoeppe]

Dependencies: #24905, #27795

[dimpase]

comment:17

How about adding checks for all the needed perl packages in the style of #27795 to polymake itself?
Actually, it would be easier (per package), as Term::ReadLine::Gnu is non-standard in the sense one cannot just use it.

As usual, they pay a usability price by not using autotools :-)

[mkoeppe]

comment:18

Replying to @dimpase:

How about adding checks for all the needed perl packages in the style of #27795 to polymake itself?

polymake itself does check for the packages needed at build time in its configure script.
The MongoDB package is checked at run time of polymake for the optional polyDB feature.

[mkoeppe]

comment:19

The challenge in this ticket is to make the installation of missing Perl packages not depend on internet access.
Here's a possible approach.

First install App::cpanminus.
Then

cpanm --local-lib-contained $PERL_LOCAL_LIB_ROOT --exclude-vendor --save-dists $PARTIAL_CPAN_MIRROR XML::Writer XML::LibXML XML::LibXSLT File::Slurp Term::ReadLine::Gnu JSON SVG MongoDB

will (1) download and install these prerequisites in $PERL_LOCAL_LIB_ROOT,
(2) save the source packages in a directory tree $PARTIAL_CPAN_MIRROR.

These are 58 packages, a total of 5 MB of source archives, which we can pack up as the source tar ball of the new spkg perl_cpan_polymake_prereq.

Then figure out how to install with plain cpan (not cpanm, which is not available out of the box on macOS) using this local partial CPAN mirror.

[mkoeppe]

Branch: u/mkoeppe/perl_packages_for_polymake

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

31b5223

Add perl_cpan_polymake_prereq as a polymake dependency

[sagetrac-git]

Commit: 31b5223

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

476c148	Install CPAN modules with cpanm
7611f43	Use cpanm --skip-satisfied
749c447	List dependencies in Makefile.PL

[sagetrac-git]

Changed commit from 31b5223 to 749c447

[mkoeppe]

Changed dependencies from #24905, #27795 to none

[mkoeppe]

Author: Matthias Koeppe

[jplab]

comment:89

Let me know when this is good to be tested: turns out, I might have more time now for testing(!)...

[mkoeppe]

comment:90

pushing these forward to 9.2

[sagetrac-git]

Changed commit from b74fc93 to e787238

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

e787238

Merge tag '9.1.rc1' into t/27763/perl_packages_for_polymake

[sagetrac-git]

Changed commit from e787238 to 20b8ffe

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

8c65337	polymake SPKG.txt: Remove outdated info regarding macOS headers
20b8ffe	Merge branch 't/29582/polymake_spkg_txt__remove_outdated_info_regarding_macos_headers' into t/27763/perl_packages_for_polymake

[mkoeppe]

Dependencies: #29582

[sagetrac-git]

Changed commit from 20b8ffe to 225dea9

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

225dea9

Merge tag '9.1.rc2' into t/27763/perl_packages_for_polymake

[mkoeppe]

comment:95

NOT ready for new tests.

[jplab]

comment:96

Replying to @mkoeppe:

NOT ready for new tests.

Ok, just I'll wait until you change the flag to needs review and then I could run some installation tests.

[mkoeppe]

comment:98

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

comment:99

Setting a new milestone for this ticket based on a cursory review.