cygwin metaticket: port Sage to Microsoft Windows (via Cygwin): stage 1 -- make building Sage automatic

[williamstein]

The goal of this ticket is that a person can:

1. Install Cygwin and certain standard Cygwin packages (listed below).

2. Extract the Sage tarball and type "make"

and have everything build automatically with no errors.

The goal is not that the resulting build works or Sage starts up (this is #13841), but if that happens as well, this will be great. Adding checks for all these prereqs, if necessary, will also be part of a future ticket.

---

More info

Most recent trials and a lot more archived status detail is at http://trac.sagemath.org/sage_trac/wiki/CygwinPort

Current instructions (work on Windows XP and Windows 7 with latest Cygwin)

As below with Sage 5.9.beta0

Cygwin prereqs

Here is what to install from Cygwin - use the usual stable binaries.

• make, perl, m4, binutils
• either gcc4-core alone and then use the optional gcc-4.7.2 spkg (the standard gcc-4.6.3 spkg won't be able to compile ecl) or gcc4-core and gcc4-g++ and gcc4-fortran whose versions must match
  • Currently, we also need to pull libmpfr4 for obscure reasons upstream that will likely be fixed soon - we hope
• lapack, liblapack-devel (this should automatically pull liblapack0, in fact, just installing liblapack-devel should already) - though we should be able to build ATLAS

Other instructions:

• Just to make sure, avoid building in home directories of Windows domain users, as they are treated in a special way by Windows (and Cygwin).
• It's a good idea that all the pathnames do not contain capital letters (Windows is case-insensitive in this way, unlike Unices), spaces, etc., see on Cygwin ECL does not handle case-sensitve paths well #13343
• Similarly, do not test without making sure that SAGE_TESTDIR does not contain spaces.
• Also, don't forget to export SAGE_PORT=yes (only needed the first time you issue make though, not after failures (memleak, rebase...) and relaunching the build)!

Spkgs

Install the following spkgs ahead of time, e.g. in SAGE_ROOT/spkg/standard/ before compiling

• Currently none

Patches

You may have to add patches during the build of the Sage library. Once it fails, do (assuming you are in SAGE_ROOT)

./sage -sh
cd devel/sage
<import the patches - you may need to make ~/.hgrc>
cd ../..
./sage -b
<assuming all goes well here>
touch spkg/installed/sage-5.6.rc0 # or whatever the version number is
exit

which will bring you back to your normal shell.

Patches currently needed for:

• Currently none

This should then work 'out of the box', modulo rebasing issues (see CygwinPort). See the wiki for other details as well.

---

There is (or was) a very old binary of Cygwin available here:

http://sage.math.washington.edu/home/wstein/tmp/sage-4.1-cygwin-i686-CYGWIN_NT-5.1.tar.gz

Depends on #14031
Depends on #14465

CC: @dimpase @mwhansen @jpflori @kcrisman @simon-king-jena

Component: porting: Cygwin

Keywords: sd31 sd32

Reviewer: Jean-Pierre Flori, Dmitrii Pasechnik, Karl-Dieter Crisman, Mike Hansen, William Stein, Luis Felipe Tabera Alonso

Issue created by migration from https://trac.sagemath.org/ticket/6743

[williamstein]

comment:1

The goal of this ticket is to port the standard Sage release to at least build and startup under Cygwin with GCC>=4.3.2.

This ticket could be closed by any significant advanced in this direction that can be merged in.

[williamstein]

Attachment: buildnotes.txt

[williamstein]

comment:2

Attachment: trac-6743.patch.gz

[williamstein]

comment:10

Attachment: trac_6743-SCRIPTS-part1.patch.gz

[williamstein]

comment:12

Important remark. I got passed a lot of issues toward startup by copying csage.so to be /bin/csage.dll. Also, this program is very helpful: http://www.dependencywalker.com/

[williamstein]

comment:13

After using the dependency thing above, and only importing half the sage library, I finally got to test out libsingular... and it works fine!

sage: R.<x,y,z> = QQ[]
sage: f = (x+y+z+1)^2; f
x^2 + 2*x*y + y^2 + 2*x*z + 2*y*z + z^2 + 2*x + 2*y + 2*z + 1
sage: type(f)
<type 'sage.rings.polynomial.multi_polynomial_libsingular.MPolynomial_libsingular'>

Another key point is one should increase the RAM available to cygwin by editing the registry, as explained in the Cygwin user's manual. The default 384MB limit with all cygwin installs is way way too small for sage. Make it a gig.

My working sage install is /home/wstein/sage-4.1 on the winxp3 virtual machine.

[jdemeyer]

comment:258

Replying to @kcrisman:

No, I didn't realize this. However, I think Sage 5.8 builds and passes most doctests, so I don't think this was really that wrong. That said, you can revert it if you need to.

Given that it doesn't even pass prereq on Sage 5.8, I did indeed move that part to "Probably will not work" to avoid confusion.

[kcrisman]

comment:259

No, I didn't realize this. However, I think Sage 5.8 builds and passes most doctests, so I don't think this was really that wrong. That said, you can revert it if you need to.

Given that it doesn't even pass prereq on Sage 5.8, I did indeed move that part to "Probably will not work" to avoid confusion.

Good point. I moved it back but with the caveat about SAGE_PORT. Since several of the other things on the "almost works" list need special packages etc., I think that this is ok.

[kcrisman]

comment:260

Apply trac_6743-doc.patch for a first attempt at a doc patch. This would satisfy my concern about libmpfr4 for now (in the meantime, the gcc4 package apparently has been "orphaned").

Is the only other issue here #14031? I sort of assumed that was the case in my patch, but of course that is just a draft and references to those things could be removed.

[kcrisman]

Changed dependencies from #14406 to #14301

[jpflori]

comment:261

Replying to @kcrisman:

Apply trac_6743-doc.patch for a first attempt at a doc patch. This would satisfy my concern about libmpfr4 for now (in the meantime, the gcc4 package apparently has been "orphaned").

This is no longer necessary as the cygwin folks finally fixed the gcc 4.5.3 dependencies. See:

• http://cygwin.com/ml/cygwin/2013-04/msg00174.html
• http://cygwin.com/ml/cygwin/2013-04/msg00170.html
  and a fresh install of mine has pulled it automatically.

Is the only other issue here #14031? I sort of assumed that was the case in my patch, but of course that is just a draft and references to those things could be removed.

[kcrisman]

comment:262

So annoying! I totally waited for a while to do this in the hopes it would happen, and it happens just after I write this.

That said, I still think most of this is worthwhile. What do you think of this updated patch?

[kcrisman]

Attachment: trac_6743-doc.patch.gz

[jdemeyer]

comment:263

Remove the comment about SAGE_TESTDIR, that variable isn't used anymore.

[jdemeyer]

comment:264

Also mention how one is supposed to run the sage-rebase_sage.bat or sage-rebase_sage.sh scripts. Something like: open a Sage shell and then type "sage-rebase_sage.sh" or equivalently

./sage --sh sage-rebase_sage.sh

[jpflori]

comment:265

Replying to @kcrisman:

Umm, no! Especially since I can't get testing to even work on XP 32 - presumably because every time I rebase, I get a different problem related to having not enough space. I wonder if deleting all the dlls in devel/sage/sage/ would work, since the ones in devel/sage/build/ are the ones that are actually (currently) used?

FYI I succesfully built Sage on a 32 bits Windows XP (within a virtual machine).
Just had to rebase once to build the doc, and tests can be run fine!

[kcrisman]

comment:266

Remove the comment about SAGE_TESTDIR, that variable isn't used anymore.

Really? What is the new way to change the test directory? (This is still necessary at times to avoid spaces, as far as I know.)

Also mention how one is supposed to run the sage-rebase_sage.bat or sage-rebase_sage.sh scripts.

I sort of purposely didn't, because I don't know how the bat one works. Maybe this ticket should depend on #14031?

FYI I succesfully built Sage on a 32 bits Windows XP (within a virtual machine). Just had to rebase once to build the doc, and tests can be run fine!

Awesome! I guess my hardware even without a VM is just not powerful enough to avoid the eternal rebase. It's really weird. It is an older machine.

[jdemeyer]

comment:267

Replying to @kcrisman:

Remove the comment about SAGE_TESTDIR, that variable isn't used anymore.

Really? What is the new way to change the test directory?

The whole concept of a "test directory" is gone.

[jdemeyer]

Changed dependencies from #14301 to #14031

[jdemeyer]

comment:269

Has anybody ever tried to make a binary Cygwin distribution with the usual command

./sage --bdist 5.9.beta5

[kcrisman]

comment:270

The whole concept of a "test directory" is gone.

I didn't realize that was part of the change in the doctest framework - I only paid tangential attention to that :(

Has anybody ever tried to make a binary Cygwin distribution with the usual command

Not yet! (Or, at least not for quite a while - maybe William knows.) That would be for another ticket :)

[jpflori]

comment:271

Replying to @jdemeyer:

Has anybody ever tried to make a binary Cygwin distribution with the usual command

./sage --bdist 5.9.beta5

I tried and it seems to have worked fine.
I've not tried to unpack the archive into another Cygwin install yet though.

[nexttime]

comment:272

SCNR.

Should we ship the info graphic with Sage (and display it on Cygwin)? XD

[jpflori]

comment:273

Replying to @jdemeyer:

Remove the comment about SAGE_TESTDIR, that variable isn't used anymore.

I've begun working on Karl-Dieter patch and came to the conclusion I'd better clean up much more of source.rst which is quite messy.
So I propose to move that to another ticket and make it a dependency here.

[jpflori]

comment:274

Replying to @jpflori:

Replying to @jdemeyer:

Has anybody ever tried to make a binary Cygwin distribution with the usual command

./sage --bdist 5.9.beta5

I tried and it seems to have worked fine.
I've not tried to unpack the archive into another Cygwin install yet though.

The archive was produced on 32 bits Windows XP within a virtual machine.
I've untarred it in another 64 bits Windows 7 still within a virtual machine.
The Cygwin setup is basically the same, which surely explains I had no rebase troubles.
And Sage seems to work fine (I've not run "make ptestlong" though).
I can post the build somewhere if someone is interested but it was built on a recent enough Core i7 so won't be that portable.

[jdemeyer]

comment:275

That's super-cool. Any chance for a Cygwin buildbot?

[jdemeyer]

comment:276

Replying to @jpflori:

I can post the build somewhere if someone is interested but it was built on a recent enough Core i7 so won't be that portable.

If you build Sage with SAGE_FAT_BINARY=yes, it should work on other processors too (which doesn't mean that it actually will, there are surely bugs with SAGE_FAT_BINARY).

[dimpase]

comment:277

Replying to @jdemeyer:

That's super-cool. Any chance for a Cygwin buildbot?

in some cases one has to rebase the whole Cygwin installation, and this can't be done from within Cygwin. So this requires more hacks,
and it is not so clear if this will work well enough.

As a first step towards this one would need to automate rebasing at the time Sage is built. This is already not 100% trivial, as fork errors can manifest themselves in different ways.

[jpflori]

Changed dependencies from #14031 to #14031, #14465

[jpflori]

comment:278

I've put the updated source.rst patch a #14465.

[jdemeyer]

comment:279

Should this ticket be closed or do we want to do that only when there is a Cygwin buildbot?

[jpflori]

comment:280

I'd say we can close it.
Of course, it does not really builds completely automatically because you have to rebase sometimes (i had to do that once for 5.8 and 3 times for 5.9) but at least thats documented and we cannot really solve this prolblem in an easy way without hacking the build system.

[jdemeyer]

Changed reviewer from Jean-Pierre Flori, Dmitrii Pasechnik, Karl-Dieter Crisman, Mike Hansen, William Stein, Luis Tabera to Jean-Pierre Flori, Dmitrii Pasechnik, Karl-Dieter Crisman, Mike Hansen, William Stein, Luis Felipe Tabera Alonso