Factor out shortcodes as pandoc-filter

[nichtich]

Quarto shortcodes are a powerful, yet non-standard, extension to pandoc markdown syntax. Having them separated as reusabe lua filter, located at https://github.com/pandoc/lua-filters or in a separate git repository would allow to use this feature independently from Quarto and help to agree on common markdown extensions instead of every tool having its own extensions. The current implementation is somwhere in the Quarto sources. I managed to use it with a Quarto installation this way:

#!/bin/bash
export QUARTO_FILTER_PARAMS=$(echo '{"results-file":"tmp.html"}' | base64)
FILTER=/opt/quarto/share/filters/quarto-pre/quarto-pre.lua
INPUT='{{< pagebreak >}} {{< meta title >}}'
pandoc -L $FILTER <(echo "$INPUT") -M title=test

result:

<p><div style="page-break-after: always;"></div> test</p>

This use is obviously fragile. I would welcome an official release of shortcodes as lua filter. I guess that it's enough to combine shortcodes.lua and shortcodes-handlers.lua and to provide an alternative to lua function call param to get the the list of custom shortcode files (could be a document meta variable or an environment variable such as FILTER_SHORTCODES). Sure I could just copy and adjust the code and add it to lua-filters but then shortcodes feature is forked and with further evolution of Quarto we likely have two slightly differing sets and types of shortcodes.

tl;dr: it makes sense to reuse shortcodes independently from Quarto but this should be made easy and stable.

[jjallaire]

Yes, it would be good to have this work in a uniform fashion. I've put up a PR to lua-filters here: pandoc/lua-filters#231

Note that I did need to extract a fair number of helper functions from Quarto's Lua code to make this standalone so it's not super-straightforward to literally share the code going forward. That said, I will track changes in both places and make sure that things stay more or less in sync.

[nichtich]

Great, thanks a lot! I guess include is not implemented in lua because technically it is not a shortcode but a preprocessor, isn't it? I can still implement a shortcode version of include-files that mimics Quarto's include with some limitations (I guess that section identifiers and link definitions can be problematic in some cases).

[jjallaire]

That's right, include is a pre-processor that we implement at the typescript level (it actually needs to run before Knitr/Jupyter so that computational cells can be included). But certainly porting include-files to a shortcode would be a fine thing to do.

[jjallaire]

It turns out that there is some ongoing discussion about what should be included in the lua-filters repo so for now its better to keep this standalone. Available here: https://github.com/quarto-dev/quarto-shortcodes