sage --docbuild: Add options to list all documents

[mkoeppe]

In contrast to the existing -D option, it would be a simple machine-readable output; and given a document such as en/reference, it would list its subdocuments.

Using this output we can reimplement the parallel build of all documents (AllBuilder._wrapper) using a simple Makefile.

CC: @jhpalmieri

Component: documentation

Author: John Palmieri

Branch/Commit: b67798e

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/31353

[jhpalmieri]

comment:1

While we're at it, we should perhaps upgrade from optparse to argparse.

[jhpalmieri]

comment:2

Something like this will allow the use of sage --docbuild -all-documents ARG, where ARG is either reference for one list or any other string for the other list:

diff --git a/src/sage_setup/docbuild/__init__.py b/src/sage_setup/docbuild/__init__.py
index f4e8522e28..77547ec6d0 100644
--- a/src/sage_setup/docbuild/__init__.py
+++ b/src/sage_setup/docbuild/__init__.py
@@ -1466,9 +1466,16 @@ def help_wrapper(option, opt_str, value, parser):
         print(help_documents(), end="")
     if option.dest == 'formats':
         print(help_formats(), end="")
+    if option.dest == 'all_documents':
+        if value == 'en/reference' or value == 'reference':
+            b = ReferenceBuilder('reference')
+            refdir = os.path.join(os.environ['SAGE_DOC_SRC'], 'en', b.name)
+            s = sorted(b.get_all_documents(refdir))
+        else:
+            s = get_documents()
+        print(s)
     setattr(parser.values, 'printed_list', 1)
 

@@ -1581,6 +1588,10 @@ def setup_parser():
     advanced.add_option("-k", "--keep-going", dest="keep_going",
                         default=False, action="store_true",
                         help="Do not abort on errors but continue as much as possible after an error")
+    advanced.add_option("--all-documents", dest="all_documents",
+                        type="str", action="callback", callback=help_wrapper,
+                        help="list ALL available DOCUMENTs, including subdocuments "
+                        "of en/reference. Mainly for internal use.")
     parser.add_option_group(advanced)
 
     return parser

[jhpalmieri]

comment:3

Sample output:

% ./sage --docbuild --all-documents reference
['reference/algebras', 'reference/arithgroup', 'reference/arithmetic_curves', 'reference/asymptotic', 'reference/calculus', 'reference/categories', 'reference/coding', 'reference/coercion', 'reference/combinat', 'reference/constants', 'reference/cpython', 'reference/cryptography', 'reference/curves', 'reference/data_structures', 'reference/databases', 'reference/diophantine_approximation', 'reference/discrete_geometry', 'reference/doctest', 'reference/dynamics', 'reference/euclidean_spaces', 'reference/finance', 'reference/finite_rings', 'reference/function_fields', 'reference/functions', 'reference/game_theory', 'reference/games', 'reference/graphs', 'reference/groups', 'reference/hecke', 'reference/history_and_license', 'reference/homology', 'reference/hyperbolic_geometry', 'reference/interfaces', 'reference/knots', 'reference/lfunctions', 'reference/libs', 'reference/logic', 'reference/manifolds', 'reference/matrices', 'reference/matroids', 'reference/misc', 'reference/modabvar', 'reference/modfrm', 'reference/modfrm_hecketriangle', 'reference/modmisc', 'reference/modsym', 'reference/modules', 'reference/monoids', 'reference/noncommutative_polynomial_rings', 'reference/number_fields', 'reference/numerical', 'reference/padics', 'reference/parallel', 'reference/plot3d', 'reference/plotting', 'reference/polynomial_rings', 'reference/power_series', 'reference/probability', 'reference/quadratic_forms', 'reference/quat_algebras', 'reference/quivers', 'reference/references', 'reference/repl', 'reference/riemannian_geometry', 'reference/rings', 'reference/rings_numerical', 'reference/rings_standard', 'reference/sat', 'reference/schemes', 'reference/semirings', 'reference/sets', 'reference/spkg', 'reference/stats', 'reference/structure', 'reference/tensor_free_modules', 'reference/valuations']
% ./sage --docbuild --all-documents whatever 
['reference', 'ja/a_tour_of_sage', 'ja/tutorial', 'it/a_tour_of_sage', 'it/faq', 'ca/intro', 'ru/tutorial', 'pt/a_tour_of_sage', 'pt/tutorial', 'hu/a_tour_of_sage', 'de/a_tour_of_sage', 'de/tutorial', 'de/thematische_anleitungen', 'fr/a_tour_of_sage', 'fr/tutorial', 'es/a_tour_of_sage', 'es/tutorial', 'a_tour_of_sage', 'developer', 'prep', 'tutorial', 'faq', 'constructions', 'website', 'installation', 'thematic_tutorials', 'tr/a_tour_of_sage']

[mkoeppe]

comment:4

Great. For using this from Makefiles it's probably better to print it one item a line without decoration such as commas

[jhpalmieri]

Branch: u/jhpalmieri/print-all-docs

[jhpalmieri]

Commit: f17f2f6

[jhpalmieri]

comment:6

Here's a branch. I did a little sorting on the document names to reflect the order in which things should get built (reference manual first, reference/references first within that).

---

New commits:

f17f2f6

trac 31353: add option to print names of all pieces of documentation,

[jhpalmieri]

Author: John Palmieri

[jhpalmieri]

comment:7

It's ready for review in terms of my code, but I don't know if it's printing in the format you want.

[mkoeppe]

comment:8

The output is perfect, thanks a lot!

But in terms of the interface, I think it would be better if passing anything that is not all or references would give an error.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8063fd8	trac 31353: add option to print names of all pieces of documentation,
799ddf2	trac 31353: only accept 'all' or 'reference' as arguments

[sagetrac-git]

Changed commit from f17f2f6 to 799ddf2

[jhpalmieri]

comment:10

Okay, how about this?

[mkoeppe]

comment:11

Perfect except for a possible typo here

+            raise ValueError("argument for --all-documents must be either 'all'"
+                             " or 'references'")

should be 'reference' I think

[sagetrac-git]

Changed commit from 799ddf2 to b67798e

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b67798e

trac 31353: only accept 'all' or 'reference' as arguments

[jhpalmieri]

comment:13

Thanks for catching that; fixed.

[jhpalmieri]

comment:14

I've opened #31366 for converting from optparse to argparse.

[mkoeppe]

Reviewer: Matthias Koeppe

[vbraun]

Changed branch from u/jhpalmieri/print-all-docs to b67798e