Rework download/extract code in sage-spkg

[jdemeyer]

The current code in sage-spkg to determine which package to install and whether to download it is buggy and too complicated. Also get rid of calls to the newest_version script, get rid of the "download again" part (why was it needed?).

Changes in behaviour:

1. When specifying a download URL, it will always download the file, even if a corresponding spkg exists locally. This allows for easier development of spkgs, because people often post updated spkgs with the same version number.
2. "sage -f package" without a version number will use the most recent (by modification time) local spkg in spkg/standard or spkg/optional. Only if no such package exists, it will try to download it. This makes it consistent with "sage -f package-x.y.z" with a version number.
3. Allow gzip compression (in addition to bzip2 and no compression) of spkgs.

Apply attachment: 12602_spkg_download.patch to the SAGE_ROOT repository.

Depends on #12479

CC: @jhpalmieri

Component: build

Author: Jeroen Demeyer

Reviewer: John Palmieri

Merged: sage-5.0.beta10

Issue created by migration from https://trac.sagemath.org/ticket/12602

[jhpalmieri]

comment:1

Questions: if you type sage -i pkg, should it first check spkg/standard (and spkg/optional, spkg/experimental) to see if there is a matching spkg, before trying to download something? Or should it check version numbers to see if the on-line file is more recent? If it checks first in spkg/..., then should there be an option to force downloading, or should we just assume that people will figure out what's going on and delete the local version if they want to download the spkg?

[jdemeyer]

comment:2

I still think the default should be to give priority to local packages.

We could support the following (-u for "upgrade"):

./sage -i -u mpfr

which would check the latest online version of mpfr, download it if needed. What do you think?

[jdemeyer]

Changed dependencies from #10479, #10579 to #12479

[jdemeyer]

Author: Jeroen Demeyer

[simon-king-jena]

comment:9

To what repository is your patch supposed to be added? It apparently does not work with devel/sage.

[jhpalmieri]

comment:11

On line 268, why cd /? The working directory remains / for a while after that, which makes me a little uncomfortable.

[jdemeyer]

comment:12

Replying to @jhpalmieri:

On line 268, why cd /?

No real reason. I just wanted to make sure that PKG_SRC is an absolute path. So doing "cd /" would give a failure if PKG_SRC had a relative path.

I changed the relevant code to the safer

# Do a final check that PKG_SRC is a file with an absolute path
cd /
if [ ! -f "$PKG_SRC" ]; then
    echo >&2 "Error: spkg file '$PKG_SRC' not found."
    echo >&2 "This shouldn't happen, it is a bug in the sage-spkg script."
    exit 1
fi

# Go back to SAGE_ROOT where we have less chance of completely messing
# up the system if we do something wrong.
cd "$SAGE_ROOT" || exit

[jdemeyer]

Attachment: 12602_spkg_download.patch.gz

[jhpalmieri]

comment:13

Okay, this latest change looks good. The code as a whole also looks good. I've been using this for a while (as part of the sage-gcc betas), and it seems to work well as part of the Sage build, when downloading optional packages (with and without version numbers specified), and when installing local packages.

[jhpalmieri]

Reviewer: John Palmieri

[jdemeyer]

Merged: sage-5.0.beta10