Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add automatic API-doc generation & deployment #183

Merged
merged 11 commits into from
May 15, 2020
59 changes: 59 additions & 0 deletions .github/workflows/doc-deployment.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Build and Deploy Documents

on: [push, pull_request]

env:
LANG: "en_US.UTF-8"
LC_ALL: "en_US.UTF-8"
HOMEBREW_DISPLAY_INSTALL_TIMES: "ON"
HOMEBREW_NO_ANALYTICS: "ON"
HOMEBREW_NO_AUTO_UPDATE: "ON"
HOMEBREW_NO_BOTTLE_SOURCE_FALLBACK: "ON"
HOMEBREW_NO_GITHUB_API: "ON"
HOMEBREW_NO_INSTALL_CLEANUP: "ON"
PIP_DISABLE_PIP_VERSION_CHECK: "ON"
PIP_NO_CLEAN: "ON"
PIP_PREFER_BINARY: "ON"
TZ: "UTC"

jobs:
Build-API-Docs:
runs-on: macos-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Full history to get tag and commit info
- name: Install Dependencies
run: |
pip3 install --prefer-binary --no-clean --disable-pip-version-check --progress-bar off lxml fypp
brew install -f --force-bottle --keep-tmp ford
type -a ford
ford --version
gfortran --version
- name: Build Docs
run: |
git fetch --all --tags
ford -r $(git describe --always) --debug API-doc-FORD-file.md
zip -vr API-docs.zip API-doc/ -x "*.DS_Store"
- name: Upload Documentation
uses: actions/upload-artifact@v2
with:
name: FORD-API-docs
path: ./API-docs.zip
- name: Broken Link Check
uses: technote-space/broken-link-checker-action@v1
with:
TARGET: file://${{ github.workspace }}/API-doc/index.html
RECURSIVE: true
ASSIGNEES: ${{ github.actor }}
- name: Deploy API Docs
uses: peaceiris/actions-gh-pages@v3
if: github.event_name == 'push' && github.repository == 'fortran-lang/stdlib' && ( startsWith( github.ref, 'refs/tags/' ) || github.ref == 'refs/heads/master' )
with:
deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}
external_repository: fortran-lang/stdlib-docs
publish_dir: ./API-doc
publish_branch: master
allow_empty_commit: true
force_orphan: false
commit_message: "From https://github.com/${{ github.repository }}/commit/${{ github.sha }} ${{ github.ref }}"
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
build/

# FORD generated documentation
# !WARNING! This folder gets deleted and overwritten
API-doc/

# Prerequisites
*.d

Expand Down
58 changes: 58 additions & 0 deletions API-doc-FORD-file.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
project: Fortran-lang/stdlib
summary: A community driven standard library for (modern) Fortran
src_dir: src
exclude_dir: src/tests
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I excluded the tests from the documentation. We could include them if we wish but processing takes longer and it might dilute the API documentation with other stuff.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with not including the tests in the API documentation. Furthermore, specs include some examples for each procedure.

output_dir: API-doc
page_dir: doc
media_dir: doc/media
fpp_extensions: fypp
preprocess: true
macro: MAXRANK=4
preprocessor: fypp
display: public
protected
private
source: true
proc_internals: true
md_extensions: markdown.extensions.toc
graph: true
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It currently takes ~20 minutes to generate the graphs. We may decide to disable this. We could set this to false and then manually use the entity metadata to enable a subset of graphs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAXRANK could be limited to 3. However, this will only marginally reduced the time. Because more procedures are expected, disabling the automatic generation of the graphs seems to be needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When building during CI waiting ~20 minutes isn't the end of the world, IMO. I've added parameters to prune/limit the depth and nodes per graph. I think it's OK to leave this in the workflow, but could be convinced otherwise.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this now done for every PR that gets submitted to stdlib?

20 minutes is not the end of the world, but it significantly slows down the development.

Do you know why it takes so long?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why it takes so long?

I guess that the issue is the generation of the graphs for all generated (hundreds?) functions in the different submodules.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also limit MAXRANK to 3 as suggested by @jvdp1. Should speed things up, and folks can extrapolate to higher rank code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zbeekman do you know how to restrict the rank? As it is now, all tests finish quickly except this documentation test that takes forever. Anything we can do to speed it up would help, at least in our initial stages. As we turn into production (we move functionality from experimental to main), we can revisit this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zbeekman do you know how to restrict the rank? As it is now, all tests finish quickly except this documentation test that takes forever. Anything we can do to speed it up would help, at least in our initial stages. As we turn into production (we move functionality from experimental to main), we can revisit this.

@certik Yes: MAXRANK=<n> here Do you want me to submit a PR restricting API documentation to public and protected entities only, reducing the MAXRANK and turning off search index and graph building unless documentation is going to be deployed? Also, what RANK should we restrict to? 2 or 3?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you could do that, that would be awesome. Can search index + graph building fail? If so, then we have to test it. Last thing we want is to fail at deploy.

Regarding rank, I don't know, try 3 and if it is fast enough, let's keep it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@certik

Can search index + graph building fail?

I've never seen search indexing fail and I don't think I've ever seen graph building fail, but this one I'm more nervous about.

graph_maxnodes: 200
graph_maxdepth: 5
coloured_edges: true
sort: permission-alpha
extra_mods: iso_fortran_env:https://gcc.gnu.org/onlinedocs/gfortran/ISO_005fFORTRAN_005fENV.html
iso_c_binding:https://gcc.gnu.org/onlinedocs/gfortran/ISO_005fC_005fBINDING.html#ISO_005fC_005fBINDING
print_creation_date: true
creation_date: %Y-%m-%d %H:%M %z
project_github: https://github.com/fortran-lang/stdlib
project_download: https://github.com/fortran-lang/stdlib/archive/master.zip
project_website: https://stdlib.fortran-lang.org
favicon: doc/media/favicon.ico
license: by-sa
author: fortran-lang/stdlib contributors
author_pic: https://fortran-lang.org/assets/img/fortran_logo_512x512.png
author_email: [email protected]
github: https://github.com/fortran-lang
twitter: https://twitter.com/fortranlang
website: https://fortran-lang.org
dbg: true
---

[TOC]

@warning This API documentation for the Fortran-lang/stdlib is a work in progress

@note
Use the navigation bar at the top of the screen to browse modules, procedures, source files, etc.
The listings near the bottom of the page are incomplete.

Fortran stdlib API Documentation
================================

This is the main API documentation landing page generated by [FORD].
The documentation for comment markup in source code, running [FORD] and the [FORD project file] are all maintained on the [FORD wiki].

[FORD]: https://github.com/Fortran-FOSS-Programmers/ford#readme
[FORD wiki]: https://github.com/Fortran-FOSS-Programmers/ford/wiki
[FORD project file]: https://github.com/fortran-lang/stdlib/blob/master/API-doc-FORD-file.md
6 changes: 5 additions & 1 deletion doc/API.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
---
title: Fortran stdlib API
---

# Fortran stdlib API

TODO
@todo expand or remove
3 changes: 3 additions & 0 deletions doc/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
---
title: Specs, examples & user docs
---
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section needs expanding/sign-posting.

Binary file added doc/media/favicon.ico
Binary file not shown.
5 changes: 5 additions & 0 deletions doc/specs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: specs
---

@todo Explain what these are, how to write them, why they're needed, etc.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section also probably needs expanding/signposting.

Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Catching and handling errors
---
title: experimental_error
---

* [`check` - Checks the value of a logical condition](#check---checks-the-value-of-a-logical-condition)
* [`error_stop` - aborts the program](#error_stop---aborts-the-program)
# Catching and handling errors

[TOC]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The anchor/backlink syntax seems to be different in python markdown, and we have the automatic TOC extension enabled, so I decided to use that rather than manually maintain a TOC.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!


## `check` - Checks the value of a logical condition

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
---
title: experimental_IO
---

# IO

* [`loadtxt` - load a 2D array from a text file](#loadtxt---load-a-2d-array-from-a-text-file)
* [`open` - open a file](#open---open-a-file)
* [`savetxt` - save a 2D array into a text file](#savetxt---save-a-2d-array-into-a-text-file)
[TOC]

## `loadtxt` - load a 2D array from a text file

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
---
title: experimental_linalg
---

# Linear Algebra

* [`diag` - Create a diagonal array or extract the diagonal elements of an array](#diag---create-a-diagonal-array-or-extract-the-diagonal-elements-of-an-array)
* [`eye` - Construct the identity matrix](#eye---construct-the-identity-matrix)
* [`trace` - Trace of a matrix](#trace---trace-of-a-matrix)
[TOC]

## `diag` - Create a diagonal array or extract the diagonal elements of an array

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
---
title: experimental_optval
---

# Default values for optional arguments

* [`optval` - fallback value for optional arguments](#optval---fallback-value-for-optional-arguments)
[TOC]

## `optval` - fallback value for optional arguments

Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
title: experimental_quadrature
---

# Numerical integration

* [`trapz` - integrate sampled values using trapezoidal rule](#trapz---integrate-sampled-values-using-trapezoidal-rule)
* [`trapz_weights` - trapezoidal rule weights for given abscissas](#trapz_weights---trapezoidal-rule-weights-for-given-abscissas)
* [`simps` - integrate sampled values using Simpson's rule (to be implemented)](#simps---integrate-sampled-values-using-simpsons-rule-to-be-implemented)
* [`simps_weights` - Simpson's rule weights for given abscissas (to be implemented)](#simps_weights---simpsons-rule-weights-for-given-abscissas-to-be-implemented)
[TOC]

## `trapz` - integrate sampled values using trapezoidal rule

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Descriptive statistics
---
title: experimental_stats
---

* [`cov` - covariance of array elements](#cov---covariance-of-array-elements)
* [`mean` - mean of array elements](#mean---mean-of-array-elements)
* [`moment` - central moments of array elements](#moment---central-moments-of-array-elements)
* [`var` - variance of array elements](#var---variance-of-array-elements)
# Descriptive statistics

[TOC]

## `cov` - covariance of array elements

Expand Down