Upgrade to Singular-3-1-7p1 #17184

jdemeyer · 2014-10-20T20:04:32Z

Singular 3-1-7p1 repackaged (see spkg-src):

http://boxen.math.washington.edu/home/jpflori/upstream/singular-3.1.7p1.tar.bz2

CC: @vbraun @nbruin @burcin

Component: packages: standard

Author: Jean-Pierre Flori

Branch/Commit: 4e287c4

Reviewer: Volker Braun

Issue created by migration from https://trac.sagemath.org/ticket/17184

The text was updated successfully, but these errors were encountered:

jdemeyer · 2014-10-20T20:12:11Z

Branch: u/jdemeyer/ticket/17184

jdemeyer · 2014-10-20T20:14:57Z

Commit: 7c3f691

jdemeyer · 2014-10-20T20:14:57Z

comment:2

boxen is down, cannot test...

New commits:

`7c3f691`	`Upgrade to Singular-3-1-7`

jpflori · 2014-10-21T05:29:54Z

comment:3

One thing I actually did not test because i just forgot: check that it still builds with SAGE_DEBUG as it is quite involved for Singular.

jpflori · 2014-10-21T07:09:22Z

comment:4

Boxen is back up (from here at least)!

sagetrac-git · 2014-10-21T08:12:26Z

Branch pushed to git repo; I updated commit sha1. New commits:

`1027b8f`	`Document templates.patch`

sagetrac-git · 2014-10-21T08:12:26Z

Changed commit from 7c3f691 to 1027b8f

kiwifb · 2014-10-21T08:23:03Z

comment:6

That's a lot of dropped patches. That, on its own, would justify the upgrade.

jpflori · 2014-10-21T09:57:37Z

comment:7

Needs some work for the debug version.
I'm on it.

jpflori · 2014-10-21T10:43:52Z

Changed commit from 1027b8f to f70af8c

jpflori · 2014-10-21T10:43:52Z

Changed branch from u/jdemeyer/ticket/17184 to u/jpflori/ticket/17184

jpflori · 2014-10-21T10:43:52Z

comment:8

Should compile ok in debug mode now, though potentially not on top of the new NTL.

New commits:

`3968376`	`Update patches for Singular 3-1-7.`
`f70af8c`	`Add new alias for Singular debug memory allocation.`

sagetrac-git · 2014-10-21T13:03:50Z

Branch pushed to git repo; I updated commit sha1. New commits:

`cdedfc6`	`Correct fuzz for Singular patches.`

sagetrac-git · 2014-10-21T13:03:50Z

Changed commit from f70af8c to cdedfc6

jdemeyer · 2014-10-21T13:26:54Z

comment:11

Without the latest debug patches, without SAGE_DEBUG=yes, together with NTL-6.2.1:

sage -t --long src/sage/schemes/plane_curves/constructor.py
**********************************************************************
File "src/sage/schemes/plane_curves/constructor.py", line 69, in sage.schemes.plane_curves.constructor.Curve
Failed example:
    C.genus()
Expected:
    1
Got:
    // ** redefining my_awesome_sage_ring **
    1
**********************************************************************
1 item had failures:
   1 of  30 in sage.schemes.plane_curves.constructor.Curve
    [29 tests, 1 failure, 1.96 s]
sage -t --long src/sage/schemes/plane_curves/curve.py
**********************************************************************
File "src/sage/schemes/plane_curves/curve.py", line 116, in sage.schemes.plane_curves.curve.Curve_generic.geometric_genus
Failed example:
    C.geometric_genus()
Expected:
          1
Got:
    // ** redefining my_awesome_sage_ring **
    1
**********************************************************************
1 item had failures:
   1 of  16 in sage.schemes.plane_curves.curve.Curve_generic.geometric_genus
    [35 tests, 1 failure, 1.65 s]
sage -t --long src/sage/rings/polynomial/multi_polynomial_ideal.py
**********************************************************************
File "src/sage/rings/polynomial/multi_polynomial_ideal.py", line 1971, in sage.rings.polynomial.multi_polynomial_ideal.?.integral_closure
Failed example:
    I.integral_closure()
Expected:
    [x^2, y^5, -x*y^3]
Got:
    [x^2, x*y^4, y^5, x*y^3]
**********************************************************************
1 item had failures:
   1 of   4 in sage.rings.polynomial.multi_polynomial_ideal.?.integral_closure
    [708 tests, 1 failure, 11.48 s]
sage -t --long src/sage/schemes/plane_curves/projective_curve.py
**********************************************************************
File "src/sage/schemes/plane_curves/projective_curve.py", line 70, in sage.schemes.plane_curves.projective_curve.ProjectiveCurve_generic.arithmetic_genus
Failed example:
    C.genus()
Expected:
    4
Got:
    // ** redefining my_awesome_sage_ring **
    4
**********************************************************************

jdemeyer · 2014-10-21T14:51:24Z

comment:12

I get these doctest failures also with the latest version of this ticket, without NTL.

jpflori · 2014-10-22T09:06:41Z

comment:14

I don't really remember who committed the "my_awesome_ring" code.
Was it Volker, or Nils, or anyone else?
I didn't have a look at the code yet, but if the author could help, that would... help!

jpflori · 2014-10-23T08:45:32Z

comment:15

Looking at the failing example in src/sage/schemes/plane_curves/constructor.py, which is a call to genus going through Singular, it seems that there are three creations of SingularFunction objects:

twice normal__lib;
then genus.
Somehow the global currRingHdl is set back to zero when the second normal__lib object is created.
This was not the case with 3.1.6.
(I'm just discovering the way Singular internally works, so my description may seem naive and wrong.)

vbraun · 2014-10-23T10:46:08Z

comment:16

Wasn't me, but if there were any changes to the genus function then hopefully they fix #15477

jpflori · 2014-10-23T14:16:28Z

comment:17

When the normal lib is loaded, currRingHdl is set back to zero.
Here is when:

#0  rSetHdl (h=0x0) at ipshell.cc:4592
#1  0x00003fffa487ce04 in iiPStart (pn=<optimized out>, v=<optimized out>) at iplib.cc:456
#2  0x00003fffa487d334 in iiMake_proc (pn=0x3fffa426dd10, pack=0x3fffa42584b8, sl=0x0) at iplib.cc:612
#3  0x00003fffa487f65c in iiRunInit (p=<error reading variable: value has been optimized out>) at iplib.cc:889
#4  iiLoadLIB (fp=0x0, libnamebuf=0x3fffffffa280 "/home/jpflori/sage.git/local/share/singular/modstd.lib", newlib=0x3fffa4270280 "modstd.lib", 
    pl=<optimized out>, autoexport=<optimized out>, tellerror=<optimized out>) at iplib.cc:941
#5  0x00003fffa487f840 in iiLibCmd (newlib=0x3fffa4270280 "modstd.lib", autoexport=<optimized out>, tellerror=<optimized out>, force=<optimized out>)
    at iplib.cc:842
#6  0x00003fffa487f3f8 in iiLoadLIB (fp=0x0, libnamebuf=0x3fffffffa4a0 "/home/jpflori/sage.git/local/share/singular/locnormal.lib", newlib=
    0x3fffa42701a0 "locnormal.lib", pl=<optimized out>, autoexport=<optimized out>, tellerror=<optimized out>) at iplib.cc:950
#7  0x00003fffa487f840 in iiLibCmd (newlib=0x3fffa42701a0 "locnormal.lib", autoexport=<optimized out>, tellerror=<optimized out>, 
    force=<optimized out>) at iplib.cc:842
#8  0x00003fffa487f3f8 in iiLoadLIB (fp=0x0, libnamebuf=0x3fffffffa6c0 "/home/jpflori/sage.git/local/share/singular/reesclos.lib", newlib=
    0x3fffa4254b20 "reesclos.lib", pl=<optimized out>, autoexport=<optimized out>, tellerror=<optimized out>) at iplib.cc:950
#9  0x00003fffa487f840 in iiLibCmd (newlib=0x3fffa4254b20 "reesclos.lib", autoexport=<optimized out>, tellerror=<optimized out>, 
    force=<optimized out>) at iplib.cc:842
#10 0x00003fffa487f3f8 in iiLoadLIB (fp=0x0, libnamebuf=0x3fffffffa8e0 "/home/jpflori/sage.git/local/share/singular/normal.lib", newlib=
    0x3fffa4254a60 "normal.lib", pl=<optimized out>, autoexport=<optimized out>, tellerror=<optimized out>) at iplib.cc:950
#11 0x00003fffa487f840 in iiLibCmd (newlib=0x3fffa4254a60 "normal.lib", autoexport=<optimized out>, tellerror=<optimized out>, force=<optimized out>)
    at iplib.cc:842
#12 0x00003fffa4082cd8 in __pyx_pf_4sage_4libs_8singular_8function_10lib (__pyx_self=<optimized out>, __pyx_v_name='normal.lib')
    at build/cythonized/sage/libs/singular/function.cpp:16752
#13 __pyx_pw_4sage_4libs_8singular_8function_11lib (__pyx_self=<optimized out>, __pyx_v_name='normal.lib')
    at build/cythonized/sage/libs/singular/function.cpp:16623
#14 0x00003fffb7ecbfb4 in call_function (oparg=<optimized out>, pp_stack=0x3fffffffab28) at Python/ceval.c:4021
#15 PyEval_EvalFrameEx (f=
    Frame 0x3fff9e5b0b60, for file /home/jpflori/sage.git/local/lib/python2.7/site-packages/sage/libs/singular/function_factory.py, line 44, in __getattr__ (self=<type unknown at remote 0x3fffa40dee90>, name='normal'), throwflag=<optimized out>) at Python/ceval.c:2679
#16 0x00003fffb7ecd7e0 in PyEval_EvalCodeEx (co=<code at remote 0x3fffa40f9430>, globals=<optimized out>, locals=<optimized out>, 
    args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, kwcount=<optimized out>, defs=0x0, defcount=0, closure=0x0)
    at Python/ceval.c:3265
#17 0x00003fffb7e1d5d4 in function_call (func=<function at remote 0x3fffa40fc2a8>, arg=(<type unknown at remote 0x3fffa40dee90>, 'normal__lib'), kw=
    0x0) at Objects/funcobject.c:526

jpflori · 2014-10-23T14:42:20Z

comment:18

And this is the new piece of code in 3-1-7 in iplib.cc around line 453:

+    if ((currRing!=NULL) &&
+      ((currRingHdl==NULL)||(IDRING(currRingHdl)!=currRing)
+       ||(IDLEV(currRingHdl)>=myynest-1)))
+    {
+      rSetHdl(rFindHdl(currRing,NULL));
+      iiLocalRing[myynest-1]=NULL;
+    }

In fact the whole block (not only the one included above) used to be around line 556 in 3-1-6.

jpflori · 2014-10-23T14:52:15Z

comment:19

In that very piece of code, Singular 4-0-1 and 3-1-7 look similar, so we might face this issue even if going for 4-0-1 directly...

jpflori · 2014-10-23T15:05:17Z

comment:20

Groumpf, I don't remmeber if anyone from the Singualr team has an account here on trac...

jpflori · 2014-11-20T13:37:13Z

comment:50

I'll give it a shot, but as Singular is a standalone process I need some dark magic to attach gdb to it and debug it.

jpflori · 2014-11-20T15:03:42Z

comment:51

Backtrace:

(gdb) bt
#0  0x00000000102e98a8 in p_Init (r=0x0) at ../kernel/pInline1.h:108
#1  0x00000000102edd6c in p_One (r=0x0) at p_polys.cc:1170
#2  0x00000000100ba578 in jjE (res=0x3ffff92fe068, v=0x3ffff92fb388) at iparith.cc:4056
#3  0x00000000100cd7b0 in iiExprArith1 (res=0x3ffff92fe068, a=0x3ffff92fb388, op=291) at iparith.cc:7992
#4  0x000000001011d390 in yyparse () at grammar.y:626
#5  0x00000000100a619c in main (argc=4, argv=0x3ffff92ff628) at tesths.cc:245

vbraun · 2014-11-20T15:06:10Z

comment:52

Run with SAGE_PEXPECT_LOG=yes, this will log to $DOT_SAGE/pexpect_logs

jpflori · 2014-11-20T15:20:28Z

comment:53

Thanks a lot for the pointer.
Here is what Singular is fed with:

...
> def sage1="redTail";
> def sage2=option(sage1);
// ** right side is not a datum, assignment ignored^M
> if(defined(sage1)){kill sage1;};if(defined(sage2)){kill sage2;};def sage3="redThrough";
> def sage4=option(sage3);
// ** right side is not a datum, assignment ignored^M
> if(defined(sage3)){kill sage3;};if(defined(sage4)){kill sage4;};def sage5="intStrategy";
> def sage6=option(sage5);
// ** right side is not a datum, assignment ignored^M
> option(get);
100663424,10321^M
> if(defined(sage5)){kill sage5;};if(defined(sage6)){kill sage6;};intvec sage7=100663424,10321;
> def sage0=2;
> print(sage0);
2^M
> def sage8=2;
> def sage9=0;
> def sage10=gen(sage9);
Singular : signal 11 (v: 3170):^M
current line:>>def sage10=gen(sage9);<<^M
Segment fault/Bus error occurred (r:1416496567)^M
please inform the authors^M
^M
// ***dError: Caught Signal 11 occured at: ^M
quit

And on relaunch:

...
> def sage11="redTail";
> def sage12=option(sage11);
// ** right side is not a datum, assignment ignored^M
> if(defined(sage11)){kill sage11;};if(defined(sage12)){kill sage12;};def sage13="redThrough";
> def sage14=option(sage13);
// ** right side is not a datum, assignment ignored^M
> if(defined(sage13)){kill sage13;};if(defined(sage14)){kill sage14;};def sage15="intStrategy";
> def sage16=option(sage15);
// ** right side is not a datum, assignment ignored^M
> option(get);
100663424,10321^M
> if(defined(sage15)){kill sage15;};if(defined(sage16)){kill sage16;};intvec sage17=100663424,10321;
> def sage10=gen(sage9);
   ? `sage9` is not defined^M
   ? error occurred in or before STDIN line 10: `def sage10=gen(sage9);`^M
> def sage18=gen();
   ? error occurred in or before STDIN line 11: `def sage18=gen();`^M
   ? last reserved name was `gen`^M
   skipping text from `;` error at token `)`^M
> def sage19=_an_element_;
   ? error occurred in or before STDIN line 12: `def sage19=_an_element_;`^M
   ? last reserved name was `def`^M
   skipping text from `;` error at token `$INVALID$`^M
> def sage20=pi;
   ? `pi` is undefined^M
   ? error occurred in or before STDIN line 13: `def sage20=pi;`^M
> def sage21=1.2;
   ? no ring active^M
   ? error occurred in or before STDIN line 14: `def sage21=1.2;`^M
> def sage22=2;
> def sage23=2;
> def sage24=sage23 * sage8;
   ? `sage8` is not defined^M
   ? error occurred in or before STDIN line 17: `def sage24=sage23 * sage8;`^M
> def sage25=2;
> print(sage25);
2^M
> def sage26=2;
> def sage27=2;
> def sage28=sage27 * sage26;
> print(sage28);
4^M
> quit

jpflori · 2014-11-20T16:13:51Z

comment:54

Ok, the issue is that currRing is NULL when pOne is called.

jpflori · 2014-11-20T16:23:55Z

comment:55

It seems Singular 3-1-6 was more careful:

> def sage8=2;
> def sage9=0;
> def sage10=gen(sage9);
   ? no ring active
   ? error occurred in or before STDIN line 14: `def sage10=gen(sage9);`

vbraun · 2014-11-20T16:26:38Z

comment:56

Looks like an upstream bug, did you post it there?

jpflori · 2014-11-20T16:33:37Z

comment:57

Not yet, still investigating, it seems the following change (here reversed) between 3-1-6 and 3-1-7 may be the issue:

diff -druN latest.orig/Singular/iparith.cc latest.new/Singular/iparith.cc
--- latest.orig/Singular/iparith.cc     2014-11-19 05:06:05.000000000 -0800
+++ latest.new/Singular/iparith.cc      2014-11-20 08:30:25.068389635 -0800
@@ -8477,7 +8477,6 @@
     sArithBase.sCmds[i].name);
     sArithBase.sCmds[i].alias=1;
   }
-  #if 0
   if (currRingHdl==NULL)
   {
     #ifdef SIQ
@@ -8493,7 +8492,6 @@
     }
     #endif
   }
-  #endif
   if (!expected_parms)
   {
     switch (tok)

jpflori · 2014-11-20T16:36:23Z

comment:58

For future ref the change was part of Singular commit:

Singular/Singular@bceefc0

jpflori · 2014-11-20T16:51:50Z

comment:59

Upstream report at:

http://www.singular.uni-kl.de/forum/viewtopic.php?f=10&t=2472

And removing the "#if 0" makes the simple test pass.
Running again the testsuite now.

sagetrac-git · 2014-11-20T16:54:06Z

Branch pushed to git repo; I updated commit sha1. New commits:

`0a5f695`	`Try to prevent Singular from segfaulting.`

sagetrac-git · 2014-11-20T16:54:06Z

Changed commit from 9b3b933 to 0a5f695

jpflori · 2014-11-20T17:40:57Z

comment:61

Things look fine for me except for timeouts which I already got before in debug mode, and two spurious errors I can not reproduce.

vbraun · 2014-11-21T14:35:14Z

comment:62

Fails on OSX with (this is a fresh install):

g++ -O2 -g  -fPIC -I.. -I/Users/buildslave-sage/slave/sage_git/build/local -pipe -I. -I.. -I/Users/buildslave-sage/slave/sage_git/build/local -I/Users/buildslave-sage/slave/sage_git/build/local/include -I/Users/buildslave-sage/slave/sage_git/build/local/include -I/Users/buildslave-sage/slave/sage_git/build/local/include   -I/Users/buildslave-sage/slave/sage_git/build/local/include -fno-implicit-templates -I.. -I/Users/buildslave-sage/slave/sage_git/build/local -DNDEBUG -DOM_NDEBUG -Dx86_64Mac_darwin -DHAVE_CONFIG_H \
	  -o Singular \
	  tesths.cc iparith.o mpsr_Tok.o claptmpl.o\
	  grammar.o scanner.o attrib.o blackbox.o eigenval_ip.o extra.o fehelp.o feOpt.o ipassign.o ipconv.o ipid.o iplib.o ipprint.o ipshell.o newstruct.o lists.o sdb.o fglm.o interpolation.o silink.o ssiLink.o s_buff.o subexpr.o janet.o wrapper.o libparse.o sing_win.o gms.o pcv.o maps_ip.o walk.o walk_ip.o cntrlc.o misc_ip.o calcSVD.o pipeLink.o Minor.o MinorProcessor.o MinorInterface.o bigintm.o pyobject_setup.o denom_list.o minpoly.o countedref.o singmathic.o semaphore.o rlimit.o slInit_Static.o mpsr_Put.o mpsr_PutPoly.o mpsr_GetPoly.o mpsr_sl.o mpsr_Get.o mpsr_GetMisc.o mpsr_Error.o ndbm.o sing_dbm.o -dynamic -L/Users/buildslave-sage/slave/sage_git/build/local/kernel -L../kernel -lkernel -L/Users/buildslave-sage/slave/sage_git/build/local/lib -L/Users/buildslave-sage/slave/sage_git/build/local/lib  -L/Users/buildslave-sage/slave/sage_git/build/local/lib -lflint -lmpfr -lmpir  -ldl -lm -lsingfac -lsingcf -lflint -lmpfr -lntl -lgmp -lreadline -ltermcap -lpthread -lm   -lomalloc  ../kernel/mmalloc.o 
ld: warning: directory not found for option '-L/Users/buildslave-sage/slave/sage_git/build/local/kernel'
duplicate symbol NTL::Mat<NTL::zz_p>::Mat()  in:
    claptmpl.o
    /Users/buildslave-sage/slave/sage_git/build/local/lib/libsingcf.a(ftmpl_inst.o)

and more like that, see http://build.sagedev.org/release/builders/%20%20fast%20Volker%20MiniMac%20%28OSX%2010.10%20x86_64%29%20full/builds/0/steps/compile_1/logs/singular

jpflori · 2014-11-21T15:59:52Z

comment:63

Can you try without templates.patch?

jpflori · 2014-11-21T16:47:20Z

comment:64

Anyway, I don't see any use of the instantiated NTL classes within the Singular directory files, so I'll just update our patches to remove the explicit instantiations in claptmpl.cc.

sagetrac-git · 2014-11-21T17:03:21Z

Changed commit from 0a5f695 to 4e287c4

sagetrac-git · 2014-11-21T17:03:21Z

Branch pushed to git repo; I updated commit sha1. New commits:

`4e287c4`	`Remove useless explicit templates instantiations in Singular.`

vbraun · 2014-11-22T17:18:32Z

Changed branch from u/jpflori/ticket/17184 to 4e287c4

jdemeyer added this to the sage-6.4 milestone Oct 20, 2014

jdemeyer added c: packages: standard labels Oct 20, 2014

jpflori added the s: needs review label Oct 21, 2014

jdemeyer added s: needs work and removed s: needs review labels Oct 21, 2014

vbraun added s: needs work and removed s: positive review labels Nov 20, 2014

jpflori added s: needs review and removed s: needs work labels Nov 20, 2014

vbraun added s: needs work and removed s: needs review labels Nov 21, 2014

jpflori added s: needs review and removed s: needs work labels Nov 21, 2014

vbraun removed the s: needs review label Nov 22, 2014

vbraun closed this as completed in b36b323 Nov 22, 2014

jpflori mentioned this issue Nov 22, 2014

Upgrade to NTL 6.2.1 #16882

Closed

Upgrade to Singular-3-1-7p1 #17184

Upgrade to Singular-3-1-7p1 #17184

Comments

jdemeyer commented Oct 20, 2014

jdemeyer commented Oct 20, 2014

jdemeyer commented Oct 20, 2014

jdemeyer commented Oct 20, 2014

jpflori commented Oct 21, 2014

jpflori commented Oct 21, 2014

sagetrac-git mannequin commented Oct 21, 2014

sagetrac-git mannequin commented Oct 21, 2014

kiwifb commented Oct 21, 2014

jpflori commented Oct 21, 2014

jpflori commented Oct 21, 2014

jpflori commented Oct 21, 2014

jpflori commented Oct 21, 2014

sagetrac-git mannequin commented Oct 21, 2014

sagetrac-git mannequin commented Oct 21, 2014

jdemeyer commented Oct 21, 2014

jdemeyer commented Oct 21, 2014

jpflori commented Oct 22, 2014

jpflori commented Oct 23, 2014

vbraun commented Oct 23, 2014

jpflori commented Oct 23, 2014

jpflori commented Oct 23, 2014

jpflori commented Oct 23, 2014

jpflori commented Oct 23, 2014

jpflori commented Nov 20, 2014

jpflori commented Nov 20, 2014

vbraun commented Nov 20, 2014

jpflori commented Nov 20, 2014

jpflori commented Nov 20, 2014

jpflori commented Nov 20, 2014

vbraun commented Nov 20, 2014

jpflori commented Nov 20, 2014

jpflori commented Nov 20, 2014

jpflori commented Nov 20, 2014

sagetrac-git mannequin commented Nov 20, 2014

sagetrac-git mannequin commented Nov 20, 2014

jpflori commented Nov 20, 2014

vbraun commented Nov 21, 2014

jpflori commented Nov 21, 2014

jpflori commented Nov 21, 2014

sagetrac-git mannequin commented Nov 21, 2014

sagetrac-git mannequin commented Nov 21, 2014

vbraun commented Nov 22, 2014