Skip to content

Commit 773ec37

Browse files
author
Release Manager
committed
Trac #33740: Add conda dev environment
We add a conda enviroment that is meant for development, i.e. includes tools that one probably needs while working on sage development such as linters, ssh (for contributing to trac) etc. Moreover, we extract the conda-related things from `docs/bootstrap` to a new `bootstrap-conda` script in order to simplify the conda workflow (see updated documentation). Github workflow run: https://github.com/sagemath/sagetrac- mirror/actions/runs/2268985510 URL: https://trac.sagemath.org/33740 Reported by: gh-tobiasdiez Ticket author(s): Tobias Diez Reviewer(s): Matthias Koeppe
2 parents 5e65c16 + 87f360c commit 773ec37

File tree

9 files changed

+126
-83
lines changed

9 files changed

+126
-83
lines changed

‎.github/workflows/ci-conda.yml

+5-11
Original file line numberDiff line numberDiff line change
@@ -43,15 +43,8 @@ jobs:
4343
bash ~/miniconda.sh -b -p $HOME/miniconda
4444
echo "CONDA=$HOME/miniconda" >> $GITHUB_ENV
4545
46-
- name: Install bootstrap prerequisites
47-
run: |
48-
export PATH="$(pwd)/build/bin:$PATH"
49-
SYSTEM=$(sage-guess-package-system)
50-
eval $(sage-print-system-package-command $SYSTEM --sudo install $(sage-get-system-packages $SYSTEM _bootstrap))
51-
52-
# Create conda environment file
53-
- name: Bootstrap
54-
run: ./bootstrap
46+
- name: Create conda environment files
47+
run: ./bootstrap-conda
5548

5649
- name: Cache conda packages
5750
uses: actions/cache@v2
@@ -80,6 +73,7 @@ jobs:
8073
shell: bash -l {0}
8174
continue-on-error: true
8275
run: |
76+
./bootstrap
8377
echo "::add-matcher::.github/workflows/configure-systempackage-problem-matcher.json"
8478
./configure --enable-build-as-root --with-python=$CONDA_PREFIX/bin/python --prefix=$CONDA_PREFIX $(for pkg in $(./sage -package list :standard: --has-file spkg-configure.m4 --has-file distros/conda.txt); do echo --with-system-$pkg=force; done)
8579
echo "::remove-matcher owner=configure-system-package-warning::"
@@ -88,8 +82,8 @@ jobs:
8882
- name: Build
8983
shell: bash -l {0}
9084
run: |
91-
pip install --no-build-isolation -v -v -e pkgs/sage-conf pkgs/sage-setup
92-
pip install --no-build-isolation -v -v -e src
85+
pip install --no-build-isolation -v -v -e ./pkgs/sage-conf ./pkgs/sage-setup
86+
pip install --no-build-isolation -v -v -e ./src
9387
env:
9488
SAGE_NUM_THREADS: 2
9589

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
/environment.yml
2424
/environment-optional.yml
2525
/src/environment.yml
26+
/src/environment-dev.yml
2627
/src/environment-optional.yml
2728

2829
/src/setup.cfg

‎.vscode/settings.json

+4
Original file line numberDiff line numberDiff line change
@@ -21,4 +21,8 @@
2121
"--doctest-modules",
2222
],
2323
"python.testing.unittestEnabled": false,
24+
"cSpell.words": [
25+
"Conda",
26+
"Cython"
27+
],
2428
}

‎Makefile

+1
Original file line numberDiff line numberDiff line change
@@ -171,6 +171,7 @@ bootstrap-clean:
171171
rm -f src/doc/en/reference/repl/*.txt
172172
rm -f environment.yml
173173
rm -f src/environment.yml
174+
rm -f src/environment-dev.yml
174175
rm -f environment-optional.yml
175176
rm -f src/environment-optional.yml
176177
rm -f src/Pipfile

‎bootstrap

+1
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,7 @@ SAGE_SPKG_FINALIZE([$pkgname], [$pkgtype], [$SPKG_SOURCE], [$SPKG_TREE_VAR])"
158158
# ONLY stderr, and to re-output the results back to stderr leaving
159159
# stdout alone. Basically we swap the two descriptors using a
160160
# third, filter, and then swap them back.
161+
./bootstrap-conda && \
161162
src/doc/bootstrap && \
162163
install_config_rpath && \
163164
aclocal -I m4 && \

‎bootstrap-conda

+78
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
#!/usr/bin/env bash
2+
3+
########################################################################
4+
# Generate auto-generated conda environment files
5+
#########################################################################
6+
7+
STRIP_COMMENTS="sed s/#.*//;"
8+
RECOMMENDED_SPKG_PATTERN="@(_recommended$(for a in $(head -n 1 build/pkgs/_recommended/dependencies); do echo -n "|"$a; done))"
9+
10+
BOOTSTRAP_PACKAGES=$(echo $(${STRIP_COMMENTS} build/pkgs/_bootstrap/distros/conda.txt))
11+
SYSTEM_PACKAGES=
12+
OPTIONAL_SYSTEM_PACKAGES=
13+
SAGELIB_SYSTEM_PACKAGES=
14+
SAGELIB_OPTIONAL_SYSTEM_PACKAGES=
15+
RECOMMENDED_SYSTEM_PACKAGES=
16+
for PKG_BASE in $(./sage --package list --has-file distros/conda.txt); do
17+
PKG_SCRIPTS=build/pkgs/$PKG_BASE
18+
SYSTEM_PACKAGES_FILE=$PKG_SCRIPTS/distros/conda.txt
19+
PKG_TYPE=$(cat $PKG_SCRIPTS/type)
20+
PKG_SYSTEM_PACKAGES=$(echo $(${STRIP_COMMENTS} $SYSTEM_PACKAGES_FILE))
21+
if [ -n "PKG_SYSTEM_PACKAGES" ]; then
22+
if [ -f $PKG_SCRIPTS/spkg-configure.m4 ]; then
23+
case "$PKG_BASE:$PKG_TYPE" in
24+
*:standard)
25+
SYSTEM_PACKAGES+=" $PKG_SYSTEM_PACKAGES"
26+
;;
27+
$RECOMMENDED_SPKG_PATTERN:*)
28+
RECOMMENDED_SYSTEM_PACKAGES+=" $PKG_SYSTEM_PACKAGES"
29+
;;
30+
*)
31+
OPTIONAL_SYSTEM_PACKAGES+=" $PKG_SYSTEM_PACKAGES"
32+
;;
33+
esac
34+
else
35+
case "$PKG_TYPE" in
36+
standard)
37+
SAGELIB_SYSTEM_PACKAGES+=" $PKG_SYSTEM_PACKAGES"
38+
;;
39+
*)
40+
SAGELIB_OPTIONAL_SYSTEM_PACKAGES+=" $PKG_SYSTEM_PACKAGES"
41+
;;
42+
esac
43+
fi
44+
fi
45+
done
46+
echo >&2 $0:$LINENO: generate conda enviroment files
47+
echo "name: sage-build" > environment.yml
48+
echo "channels:" >> environment.yml
49+
echo " - conda-forge" >> environment.yml
50+
echo " - nodefaults" >> environment.yml
51+
echo "dependencies:" >> environment.yml
52+
for pkg in $SYSTEM_PACKAGES; do
53+
echo " - $pkg" >> environment.yml
54+
done
55+
echo " # Packages needed for ./bootstrap" >> environment.yml
56+
for pkg in $BOOTSTRAP_PACKAGES; do
57+
echo " - $pkg" >> environment.yml
58+
done
59+
sed 's/name: sage-build/name: sage/' environment.yml > src/environment.yml
60+
for pkg in $SAGELIB_SYSTEM_PACKAGES; do
61+
echo " - $pkg" >> src/environment.yml
62+
done
63+
sed 's/name: sage/name: sage-dev/' src/environment.yml > src/environment-dev.yml
64+
echo " # Additional dev tools" >> src/environment-dev.yml
65+
echo " - openssh" >> src/environment-dev.yml
66+
echo " - pycodestyle" >> src/environment-dev.yml
67+
echo " - pytest" >> src/environment-dev.yml
68+
69+
cp environment.yml environment-optional.yml
70+
echo " # optional packages" >> environment-optional.yml
71+
for pkg in $OPTIONAL_SYSTEM_PACKAGES; do
72+
echo " - $pkg" >> environment-optional.yml
73+
done
74+
cp src/environment.yml src/environment-optional.yml
75+
echo " # optional packages" >> src/environment-optional.yml
76+
for pkg in $OPTIONAL_SYSTEM_PACKAGES $SAGELIB_OPTIONAL_SYSTEM_PACKAGES; do
77+
echo " - $pkg" >> src/environment-optional.yml
78+
done

‎build/pkgs/jupyter_sphinx/distros/conda.txt

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
jupyter_sphinx

‎src/doc/bootstrap

+7-34
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ shopt -s extglob
2727

2828
RECOMMENDED_SPKG_PATTERN="@(_recommended$(for a in $(head -n 1 build/pkgs/_recommended/dependencies); do echo -n "|"$a; done))"
2929

30-
for SYSTEM in arch debian fedora cygwin homebrew conda; do
30+
for SYSTEM in arch debian fedora cygwin homebrew; do
3131
SYSTEM_PACKAGES=
3232
OPTIONAL_SYSTEM_PACKAGES=
3333
SAGELIB_SYSTEM_PACKAGES=
@@ -63,40 +63,13 @@ for SYSTEM in arch debian fedora cygwin homebrew conda; do
6363
fi
6464
fi
6565
done
66-
if [ "${SYSTEM}" = "conda" ]; then
67-
if [ "${BOOTSTRAP_QUIET}" = "no" ]; then
68-
echo >&2 $0:$LINENO: installing environment.yml, environment-optional.yml, src/environment.yml and src/environment-optional.yml
69-
fi
70-
echo "name: sage-build" > environment.yml
71-
echo "channels:" >> environment.yml
72-
echo " - conda-forge" >> environment.yml
73-
echo " - nodefaults" >> environment.yml
74-
echo "dependencies:" >> environment.yml
75-
for pkg in $SYSTEM_PACKAGES; do
76-
echo " - $pkg" >> environment.yml
77-
done
78-
sed 's/name: sage-build/name: sage/' environment.yml > src/environment.yml
79-
for pkg in $SAGELIB_SYSTEM_PACKAGES; do
80-
echo " - $pkg" >> src/environment.yml
81-
done
82-
cp environment.yml environment-optional.yml
83-
echo " # optional packages" >> environment-optional.yml
84-
for pkg in $OPTIONAL_SYSTEM_PACKAGES; do
85-
echo " - $pkg" >> environment-optional.yml
86-
done
87-
cp src/environment.yml src/environment-optional.yml
88-
echo " # optional packages" >> src/environment-optional.yml
89-
for pkg in $OPTIONAL_SYSTEM_PACKAGES $SAGELIB_OPTIONAL_SYSTEM_PACKAGES; do
90-
echo " - $pkg" >> src/environment-optional.yml
91-
done
92-
else
93-
if [ "${BOOTSTRAP_QUIET}" = "no" ]; then
94-
echo >&2 $0:$LINENO: installing "$OUTPUT_DIR"/$SYSTEM"*.txt"
95-
fi
96-
echo "$(sage-print-system-package-command $SYSTEM --prompt --sudo install $(echo $(echo $SYSTEM_PACKAGES | xargs -n 1 echo | sort | uniq)))" > "$OUTPUT_DIR"/$SYSTEM.txt
97-
echo "$(sage-print-system-package-command $SYSTEM --prompt --sudo install $(echo $(echo $OPTIONAL_SYSTEM_PACKAGES | xargs -n 1 echo | sort | uniq)))" > "$OUTPUT_DIR"/$SYSTEM-optional.txt
98-
echo "$(sage-print-system-package-command $SYSTEM --prompt --sudo install $(echo $(echo $RECOMMENDED_SYSTEM_PACKAGES | xargs -n 1 echo | sort | uniq)))" > "$OUTPUT_DIR"/$SYSTEM-recommended.txt
66+
67+
if [ "${BOOTSTRAP_QUIET}" = "no" ]; then
68+
echo >&2 $0:$LINENO: installing "$OUTPUT_DIR"/$SYSTEM"*.txt"
9969
fi
70+
echo "$(sage-print-system-package-command $SYSTEM --prompt --sudo install $(echo $(echo $SYSTEM_PACKAGES | xargs -n 1 echo | sort | uniq)))" > "$OUTPUT_DIR"/$SYSTEM.txt
71+
echo "$(sage-print-system-package-command $SYSTEM --prompt --sudo install $(echo $(echo $OPTIONAL_SYSTEM_PACKAGES | xargs -n 1 echo | sort | uniq)))" > "$OUTPUT_DIR"/$SYSTEM-optional.txt
72+
echo "$(sage-print-system-package-command $SYSTEM --prompt --sudo install $(echo $(echo $RECOMMENDED_SYSTEM_PACKAGES | xargs -n 1 echo | sort | uniq)))" > "$OUTPUT_DIR"/$SYSTEM-recommended.txt
10073
done
10174

10275
OUTPUT_DIR="src/doc/en/reference/spkg"

‎src/doc/en/installation/conda.rst

+28-38
Original file line numberDiff line numberDiff line change
@@ -65,25 +65,22 @@ conda-forge/sage-feedstock <https://github.com/conda-forge/sage-feedstock/issues
6565
Using conda to provide system packages for the Sage distribution
6666
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
6767

68-
If Conda is installed (check by typing ``conda info``), there are two ways to
69-
prepare for installing SageMath from source:
68+
If Conda is installed (check by typing ``conda info``), one can install SageMath
69+
from source as follows:
7070

7171
- If you are using a git checkout::
7272

73-
$ ./bootstrap
73+
$ ./bootstrap-conda
7474

75-
- Create a new empty environment and activate::
75+
- Create a new conda environment including all standard packages
76+
recognized by sage, and activate it::
7677

77-
$ conda create -n sage-build
78+
$ conda env create --file environment.yml --name sage-build
7879
$ conda activate sage-build
7980

80-
- Install standard packages recognized by sage's ``spkg-configure`` mechanism::
81-
82-
$ conda env update --file environment.yml -n sage-build
83-
84-
- Or install all standard and optional packages recognized by sage::
85-
86-
$ conda env update --file environment-optional.yml -n sage-build
81+
Alternatively, use ``environment-optional.yml`` in place of
82+
``environment.yml`` to create an environment with all standard and optional
83+
packages recognized by sage.
8784

8885
- Then the SageMath distribution will be built using the compilers provided by Conda
8986
and using many packages installed by Conda::
@@ -114,38 +111,34 @@ Here we assume that you are using a git checkout.
114111

115112
$ export SAGE_NUM_THREADS=24
116113

117-
- As a recommended step, install the ``mamba`` package manager. If
114+
- As a recommended step, install the ``mamba`` package manager. If
118115
you skip this step, replace ``mamba`` by ``conda`` in the
119116
following steps::
120117

121118
$ conda install mamba
122119

123-
- Create and activate a new conda environment that provides the
124-
bootstrapping prerequisites. You can replace 3.9 by another Python
125-
version::
126-
127-
$ mamba create -n sage-build python=3.9 \
128-
gettext autoconf automake libtool pkg-config
129-
$ conda activate sage-build
130-
131-
- Run ``bootstrap``; this generates the files ``src/environment*.yml`` used
120+
- Generate the conda environment files ``src/environment*.yml`` used
132121
in the next step::
133122

134-
$ ./bootstrap
135-
136-
- Populate the conda environment with the dependencies of Sage::
123+
$ ./bootstrap-conda
137124

138-
$ mamba env update -n sage-build -f src/environment.yml # alternatively, use
125+
- Create and activate a new conda environment with the dependencies of Sage
126+
and a few additional developer tools::
139127

140-
Alternatively, you can use ``src/environment-optional.yml``, which will
141-
install some additional packages.
128+
$ mamba env create --file src/environment-dev.yml --name sage-dev
129+
$ conda activate sage-dev
142130

143-
- Activate the conda environment again::
131+
Alternatively, you can use ``src/environment.yml`` or
132+
``src/environment-optional.yml``, which will only install standard
133+
(and optional) packages without any additional developer tools.
144134

145-
$ conda activate sage-build
135+
By default, the most recent version of Python supported by Sage is
136+
installed. You can use the additional option ``python=3.9`` in the above
137+
``env create`` command to select another Python version (here 3.9).
146138

147139
- Run the ``configure`` script::
148140

141+
$ ./bootstrap
149142
$ ./configure --with-python=$CONDA_PREFIX/bin/python \
150143
--prefix=$CONDA_PREFIX \
151144
$(for pkg in $(./sage -package list :standard: \
@@ -154,20 +147,17 @@ Here we assume that you are using a git checkout.
154147
echo --with-system-$pkg=force; \
155148
done)
156149

157-
- Install the build prerequisites of the Sage library::
158-
159-
$ pip install --no-build-isolation -v -v --editable pkgs/sage-conf pkgs/sage-setup
160-
161-
- Install the Sage library::
150+
- Install the build prerequisites and the Sage library::
162151

163-
$ pip install --no-build-isolation -v -v --editable src
152+
$ pip install --no-build-isolation -v -v --editable ./pkgs/sage-conf ./pkgs/sage-setup
153+
$ pip install --no-build-isolation -v -v --editable ./src
164154

165155
- Verify that Sage has been installed::
166156

167157
$ sage -c 'print(version())'
168158
SageMath version 9.6.beta5, Release Date: 2022-03-12
169159

170-
Note that ``make`` is not used at all. All dependencies
160+
Note that ``make`` is not used at all. All dependencies
171161
(including all Python packages) are provided by conda.
172162

173163
Thus, you will get a working version of Sage much faster. However,
@@ -179,6 +169,6 @@ library is installed in editable mode. This means that when you only
179169
edit Python files, there is no need to rebuild the library; it
180170
suffices to restart Sage.
181171

182-
After editing any Cython files, rebuild by repeating the command::
172+
After editing any Cython files, rebuild the Sage library using::
183173

184174
$ pip install --no-build-isolation -v -v --editable src

0 commit comments

Comments
 (0)