Upgrade to Singular-4.x.x #17254

jdemeyer · 2014-10-29T15:17:10Z

Lots of stuff has changed in Singular 4.
This is a big update.

Build system completely changed, and a lot of internal stuff.

Original upstream tarball at www.mathematik.uni-kl.de/ftp/pub/Math/Singular/SOURCES/4-0-3/: singular-4.0.3p3 dot tar dot gz

Modified tarball made with spkg-src at:

http://www.lmona.de/files/sage/singular-4.0.3p3.tar.bz2

This corresponds to commit a070b84d whose message tells 4.0.3p3 and on which this branch was based:

Singular/Singular@a070b84

You can also make your own tarball based on this commit or another one with

env SINGULAR_GIT_COMMIT=a070b84d ./sage --sh build/pkgs/singular/spkg-src
./sage --package fix-checksum singular

This script is loosely based on the make_tar.sh script from upstream.

Depends on #17635

CC: @jpflori @burcin @sagetrac-jakobkroeker @kiwifb @malb @mkoeppe @vbraun @nthiery @bhutz @miguelmarco @sagetrac-gjorgenson

Component: packages: standard

Author: Jakob Kroeker, Jean-Pierre Flori, Jeroen Demeyer, John Perry, François Bissey, Leif Leonhardy, Dima Pasechnik

Branch: f1a0dcc

Reviewer: François Bissey, Jeroen Demeyer, Ben Hutz, Leif Leonhardy, Dima Pasechnik, Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/17254

The text was updated successfully, but these errors were encountered:

jdemeyer · 2014-10-29T19:30:43Z

Branch: u/jdemeyer/ticket/17254

jdemeyer · 2014-10-29T19:30:43Z

Commit: 2fcaeb5

jpflori · 2014-10-30T10:41:39Z

comment:2

Yeah, the update is non trivial, that's a reason to get #17184 (and so #16882) inbetween.

jdemeyer · 2014-10-30T10:53:28Z

comment:3

Replying to @jpflori:

Yeah, the update is non trivial

I wonder if we even have a Sage developer who is familiar enough with Singular to do this (for the record: not me, I know nothing about Singular).

that's a reason to get #17184 (and so #16882) inbetween.

Of course, I'm not arguing with that.

jpflori · 2014-10-30T13:02:07Z

comment:4

Replying to @jdemeyer:

Replying to @jpflori:

Yeah, the update is non trivial

I wonder if we even have a Sage developer who is familiar enough with Singular to do this (for the record: not me, I know nothing about Singular).

Maybe Burcin would do...
IIRC the update was proposed as a GSoC project.

sagetrac-jakobkroeker · 2014-11-03T11:59:40Z

comment:6

Q: where to get the singular tarball for this commit?
I get a failure at

Trying to download http://www.sagemath.org/packages/upstream/singular/singular-4.0.1p1.tar.bz2

Update to Singular 4.0.1

Maybe Burcin would do...

Could the update to 4.0.1. be done by at least two or three persons to spread the knownledge
about Singular internals?

jpflori · 2014-11-03T12:34:18Z

comment:7

You have to put a tarball repackaged by the spkg-src script in the upstream folder directly.
Once the ticket is merged, the tarball will be put on the official mirror and could be downloaded automatically but not at the moment.

I've put a repackaged tarball at:

http://boxen.math.washington.edu/home/jpflori/upstream/singular-4.0.1.tar.bz2

jpflori · 2014-11-03T12:35:18Z

comment:8

(I'm not sure the checksums will the same as what Jeroen branch expect, run ./sage -sh sage-fix-pkg-checksums after you have downloaded the tarball to fix this.)

jpflori · 2014-11-03T12:36:03Z

comment:9

And I'd be happy to work on the upgrade and learn more about Singular's internals, but please also review the "easy" #17184 (where I've already learnt some stuff) firsT.

sagetrac-jakobkroeker · 2015-01-21T10:50:46Z

comment:11

Singular 4.0.1. compiles now in sage but sage does not yet -
I just pushed a branch 'u/jakobkroeker/ticket/17254'.
You have to correct the 'singular-4.0.1p1.tar.bz2' tarball to get it work ( the 'm4' folder is missing).

For the moment I removed some patches; we probably will have to adjust them later on. For example there is meanwhile no such thing like 'lomalloc_ndebug' so if we want to disable some debug functionality or output for omalloc we have to do that by passing appropriate flags to omalloc/configure, e,g
'--enable-optimizationflags --disable-debug --without-debug' ?

Also I disabled the currently broken
'--enable-debugoutput' configure flag for the standalone 'factory'
and informed Oleksander Motsak about the issue.

Could someone of you take care of the sage compile errors and figure out what is going wrong and/or ask for help in the Singular forum? I can't do that easily, because within the Singular-team I'm out-of-favor because I constantly criticise their software and thus I will likely get little or no support.

Jakob

jdemeyer · 2015-01-21T13:02:53Z

Changed branch from u/jdemeyer/ticket/17254 to u/jakobkroeker/ticket/17254

jdemeyer · 2015-01-21T13:02:53Z

New commits:

`4c0bba3`	`singular 4.0.1 builds now, but sage does not due to changes in libsingular?`

jdemeyer · 2015-01-21T13:02:53Z

Changed commit from 2fcaeb5 to 4c0bba3

jdemeyer · 2015-01-21T13:04:00Z

Changed commit from 4c0bba3 to 2fcaeb5

jdemeyer · 2015-01-21T13:04:00Z

Changed branch from u/jakobkroeker/ticket/17254 to u/jdemeyer/ticket/17254

sagetrac-jakobkroeker · 2015-01-21T13:37:05Z

comment:14

first hint: for example the interface of 'naIsZero'
seems changed from

BOOLEAN naIsZero(number a)

to

BOOLEAN naIsZero(number a, const coeffs cf)

Persons who might have an idea about the changes:

Oleksandr Motsak
Mohamed Barakat
Martin Lee
Hans Schönemann

see contact details at
http://www.mathematik.uni-kl.de/agag/mitglieder/

jdemeyer · 2015-01-21T21:39:31Z

comment:16

Replying to @jpflori:

I've put a repackaged tarball at:

http://boxen.math.washington.edu/home/jpflori/upstream/singular-4.0.1.tar.bz2

That's an older version than the one that I put in the ticket description, any reason...?

jdemeyer · 2015-01-21T21:40:25Z

comment:17

Replying to @sagetrac-jakobkroeker:

You have to correct the 'singular-4.0.1p1.tar.bz2' tarball to get it work ( the 'm4' folder is missing).

My tarball compiled fine, so what is the problem?

jdemeyer · 2015-01-21T21:57:55Z

comment:18

You should never run ./autogen.sh, that is supposed to be run at packaging-time, not build-time.

sagetrac-git · 2015-01-21T22:00:30Z

Changed commit from 2fcaeb5 to 80a6b05

sagetrac-git · 2015-01-21T22:00:30Z

Branch pushed to git repo; I updated commit sha1. New commits:

`79c079f`	`Upgrade to Singular-4-0-1`
`4c0bba3`	`singular 4.0.1 builds now, but sage does not due to changes in libsingular?`
`80a6b05`	`Merge commit '4c0bba3' into ticket/17254`

sagetrac-jakobkroeker · 2015-01-23T16:38:59Z

comment:20

My tarball compiled fine, so what is the problem?

Confirm; cannot reproduce the issue.
So likely I didn't use the recent tarball, or I screwed something up.

You should never run ./autogen.sh, that is supposed to be run at packaging-time, not build-time.

I'm guilty ;-)

sagetrac-jakobkroeker · 2015-01-26T17:43:32Z

comment:21

singulars spkg-install:

the '--enable-debugoutput' configure flag
for standalone factory needs now in additon '--enable-streamio --without-Singular'

sagetrac-jakobkroeker · 2015-02-02T12:06:43Z

comment:22

After a little more tweaking and temporarily disabling the extension fields/rings,
I was able to build some parts of sage with Singular 4.0.1 on a 'fc18 system' and
then.. sage crashes at start! That happens because I didn't update ring creation right - I only recently discovered examples at './libpolys/tests/rings_test.h'. And some sage packages create common rings at loading time.

remaining riddles:

during build one sage file tried to link to -lsingular (I have no idea why), instead to the libSingular* files (that changed!), so I worked around by manually creating links to libSingular* files.
on a ScientificLinux system I get errors like "ImportError: /home/kroeker/Projects/sage-singular-4.0.1/local/lib/python2.7/site-packages/sage/rings/polynomial/plural.so: undefined symbol: wFunctionalBuch". This symbol is in libSingular*
fix ring creation. See examples at './libpolys/tests/rings_test.h'
the extension fields will need some more love. See 'algext.h' and 'transext.h'. @mantepse: could you take a look?
some mapping functions (e.g. nr2mMapZp) now require a source and dst (coefficient) parameter. Not sure if I did it right (source: currentRing, dest: _ring or vice versa?)
I didn't fix unused interface definitions in singular-cdefs.pxi. TODO

See branch u/jakobkroeker/ticket/17254
and some notes at jakobkroeker/jk_test_sage#13

If a build get stuck before errors described above, then maybe a 'make distclean' is necessary.
And of course, the Singular tarball http://boxen.math.washington.edu/home/jdemeyer/spkg/singular-4.0.1p1.tar.bz2 is required.

Any help appreciated.

New commits:

`79c079f`	`Upgrade to Singular-4-0-1`
`2fcaeb5`	`Update Singular to 4.0.1`
`4c0bba3`	`singular 4.0.1 builds now, but sage does not due to changes in libsingular?`
`80a6b05`	`Merge commit '4c0bba3' into ticket/17254`
`2ae243a`	`messy fixes`
`ff2a9dc`	`required changes for singular to build sage; should be fixed upstream`
`3b7877e`	`hacking sage to build with Singular 4.0.1. needs more love for extension coefficient fields and a refactoring`
`f178dfc`	`fixing nr2mMapZp call`

sagetrac-jakobkroeker · 2016-10-02T10:20:56Z

Changed reviewer from François Bissey, Jeroen Demeyer, Ben Hutz, Leif Leonhardy, Dima Pasechnik to François Bissey, Jeroen Demeyer, Ben Hutz, Leif Leonhardy, Dima Pasechnik, Travis Scrimshaw

jpflori · 2016-10-03T10:06:03Z

comment:537

For debug mode: Better debug mode for singular 4 #21624
For linking order: Strange linking order for omalloc due to pkgconfig bug #21625
I did not open a follow up ticket for the next upgrade for obvious reasons.

sagetrac-jakobkroeker · 2016-10-03T17:09:21Z

comment:538

Replying to @jpflori:

Replying to @kiwifb:

That would be a nice idea but I guess there may still be a need for running autoconf at some point.

Sure there is.
Actually upstream instruction on using xalloc are
README for xalloc: a debug helper and alternative to omalloc
...

}}}

These instructions are buggy :)
It is enough to cd to the xalloc dir, then autoreconf and if you want debug mode, then you delete the omalloc dir and mv xalloc omalloc.

To only use patch on top of the upstream 4.0.3p3 what you need to do is to autoreconf everything on your machine on a pristine tree and on a patched one and produce the patch.

Is it possible to change the upstream source in a way that we do not need an autoreconf for xalloc? For example, would it be sufficient to add

AC_CONFIG_SUBDIRS([xalloc])

to the configure.ac at upstream?

Whatever is required to get rid of repackaging/autoreconf, we should do that.

vbraun · 2016-10-03T17:47:54Z

comment:539

There is already a different .p3 tarball on the mirrors, so we first have to propagate an updated tarball. And the master fileserver is currently down.

sagetrac-jakobkroeker · 2016-10-03T18:59:47Z

comment:540

Replying to @jpflori:

For debug mode: Better debug mode for singular 4 #21624

For linking order: Strange linking order for omalloc due to pkgconfig bug #21625

I did not open a follow up ticket for the next upgrade for obvious reasons.

There is now a metaticket #21632 to collect the remaining issues.

sagetrac-jakobkroeker · 2016-10-03T20:28:41Z

comment:541

I'm sorry, but since this thread is read by some Singular interface experts,
I would like to advertise the tickets #17696, #13999, #17697, #10708, #12529.

Could you have a look?

sagetrac-jakobkroeker · 2016-10-04T19:58:41Z

comment:542

now xalloc folder should be autoconfigured upstream,
(if I did it right);
see Singular/Singular#795

sagetrac-jakobkroeker · 2016-10-04T22:17:33Z

comment:543

I hacked into the patchbot allowing unsafe tickets and got the following:

> Successfully unpacked
Sha1 of singular-4.0.3p3.tar.bz2 is dd68e84ef7d0e6c35994e0fce0faa8edf38a7bb1
Traceback (most recent call last):
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1046, in test_a_ticket
    self.check_spkg(spkg)
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1227, in check_spkg
    given_sha = open(path).read().splitlines()[1].split('=')[1]
IOError: [Errno 2] No such file or directory: 'build/pkgs/singular/checksums.ini'
http://www.lmona.de/files/sage/singular-4.0.3p3.tar.bz2 -- 26 seconds
Apply -- 7751 seconds
http://www.lmona.de/files/sage/singular-4.0.3p3.tar.bz2 -- 26 seconds
2016-10-04 21:51:04
7777 seconds
[2016-10-04 21:51:06] Reporting #17254 with status Spkg
fatal: Unable to read current working directory: Datei oder Verzeichnis nicht gefunden
Traceback (most recent call last):
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1328, in report_ticket
    tags = [describe_branch('patchbot/base', tag_only=True),
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/util.py", line 235, in describe_branch
    universal_newlines=True)
  File "/usr/lib/python2.7/subprocess.py", line 544, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['git', 'describe', '--tags', '--match', '[0-9].[0-9]*', 'patchbot/base']' returned non-zero exit status 128
[2016-10-04 21:51:06] #17254: Spkg
REPORT
{'base': '7.4.beta6',
 'deps': [17635],
 'machine': ['Ubuntu',
             '12.04',
             'x86_64',
             '3.2.0-70-generic',
             'x200t-ThinkPad-X200-Tablet'],
 'owner': 'unknown owner',
 'patchbot_version': '2.6.3',
 'plugins': [],
 'spkgs': [u'http://www.lmona.de/files/sage/singular-4.0.3p3.tar.bz2'],
 'status': 'Spkg',
 'time': '2016-10-04 21:51:06',
 'user': 'jakobkroeker'}
17254: Spkg
ok (report successfully posted)
[2016-10-04 21:51:15] Done reporting #17254
Traceback (most recent call last):
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1509, in main
    patchbot.test_a_ticket(ticket)
  File "/home/jakobkroeker/.local/lib/python2.7/site-packages/sage_patchbot/patchbot.py", line 1185, in test_a_ticket
    shutil.rmtree(maybe_temp_root)
  File "/usr/lib/python2.7/shutil.py", line 237, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "/usr/lib/python2.7/shutil.py", line 235, in rmtree
    names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/tmp/tmp8iZiiy-sage-git-temp-17254'

vbraun · 2016-10-21T07:04:36Z

Changed branch from public/singular4 to f1a0dcc

jdemeyer · 2016-10-21T08:10:34Z

comment:545

Finally! Where is the party?

jdemeyer · 2016-10-21T08:10:34Z

Changed commit from f1a0dcc to none

kiwifb · 2016-10-21T08:41:55Z

comment:546

Replying to @jdemeyer:

Finally! Where is the party?

I could invite you to my place but I gather it is a bit far for everyone else :) I'll be experimenting with 4.0.3p4 and ntl-10 once I have absorbed 7.5.beta0.

vbraun · 2016-10-21T09:38:42Z

comment:547

It would be nice if the debug mode (#21624) could be fixed as that is tested on the buildbot...

jdemeyer added this to the sage-6.4 milestone Oct 29, 2014

jdemeyer added c: packages: standard labels Oct 29, 2014

This comment has been minimized.

Sign in to view

vbraun modified the milestones: sage-7.4, sage-7.5 Oct 3, 2016

vbraun removed the s: positive review label Oct 21, 2016

vbraun closed this as completed in d34fb70 Oct 21, 2016

sagetrac-mariah mannequin mentioned this issue Jan 31, 2020

quotient of ideals throws up Singular bug #12803

Closed

jdemeyer mentioned this issue Mar 19, 2018

Incomplete multivariate factorization #17251

Closed

This was referenced Dec 24, 2016

bug in Singulars groebner basis computation over integers #17676

Closed

stopgap for #17680 #17681

Closed

jdemeyer mentioned this issue Jun 1, 2016

Gracefully downgrade Singular 4 -> 3 #20728

Closed

a-andre mentioned this issue Jul 3, 2016

singular 3.1.7 fails to build using GCC 6 #20738

Closed

sagetrac-jakobkroeker mannequin mentioned this issue Dec 29, 2022

follow-up for upgrade to Singular 4: reuse ZZ and QQ #21619

Open

This was referenced May 2, 2017

Better debug mode for singular 4 #21624

Open

Strange linking order for omalloc due to pkgconfig bug #21625

Closed

This was referenced Nov 12, 2016

Update Singular to 4.0.3p4 #21631

Closed

Meta ticket: follow-up tickets for upgrade to Singular4 #21632

Open

complete documentation for Singular interface related functions #21633

Open

EmmanuelCharpentier mannequin mentioned this issue Mar 19, 2017

giacpy_sage doctest failure and new libsingular cf.type == n_unknown ring test #21884

Closed

Upgrade to Singular-4.x.x #17254

Upgrade to Singular-4.x.x #17254

Comments

jdemeyer commented Oct 29, 2014

jdemeyer commented Oct 29, 2014

jdemeyer commented Oct 29, 2014

This comment has been minimized.

jpflori commented Oct 30, 2014

jdemeyer commented Oct 30, 2014

jpflori commented Oct 30, 2014

sagetrac-jakobkroeker mannequin commented Nov 3, 2014

jpflori commented Nov 3, 2014

jpflori commented Nov 3, 2014

jpflori commented Nov 3, 2014

This comment has been minimized.

sagetrac-jakobkroeker mannequin commented Jan 21, 2015

jdemeyer commented Jan 21, 2015

jdemeyer commented Jan 21, 2015

jdemeyer commented Jan 21, 2015

jdemeyer commented Jan 21, 2015

jdemeyer commented Jan 21, 2015

sagetrac-jakobkroeker mannequin commented Jan 21, 2015

jdemeyer commented Jan 21, 2015

jdemeyer commented Jan 21, 2015

jdemeyer commented Jan 21, 2015

sagetrac-git mannequin commented Jan 21, 2015

sagetrac-git mannequin commented Jan 21, 2015

sagetrac-jakobkroeker mannequin commented Jan 23, 2015

sagetrac-jakobkroeker mannequin commented Jan 26, 2015

sagetrac-jakobkroeker mannequin commented Feb 2, 2015

sagetrac-jakobkroeker mannequin commented Oct 2, 2016

jpflori commented Oct 3, 2016

sagetrac-jakobkroeker mannequin commented Oct 3, 2016

vbraun commented Oct 3, 2016

sagetrac-jakobkroeker mannequin commented Oct 3, 2016

sagetrac-jakobkroeker mannequin commented Oct 3, 2016

sagetrac-jakobkroeker mannequin commented Oct 4, 2016

sagetrac-jakobkroeker mannequin commented Oct 4, 2016

vbraun commented Oct 21, 2016

jdemeyer commented Oct 21, 2016

jdemeyer commented Oct 21, 2016

kiwifb commented Oct 21, 2016

vbraun commented Oct 21, 2016