Skip to content

Commit 231d210

Browse files
authored
[ new ] golden testing framework (agda#1518)
1 parent aa12b7d commit 231d210

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

80 files changed

+2775
-51
lines changed

.github/workflows/ci-ubuntu.yml

+14-19
Original file line numberDiff line numberDiff line change
@@ -68,12 +68,12 @@ jobs:
6868
if [[ '${{ github.ref }}' == 'refs/heads/master' \
6969
|| '${{ github.base_ref }}' == 'master' ]]; then
7070
# Pick Agda version for master
71-
echo "AGDA_COMMIT=tags/v2.6.1.3.20210524" >> $GITHUB_ENV;
71+
echo "AGDA_COMMIT=tags/v2.6.2" >> $GITHUB_ENV;
7272
echo "AGDA_HTML_DIR=html" >> $GITHUB_ENV
7373
elif [[ '${{ github.ref }}' == 'refs/heads/experimental' \
7474
|| '${{ github.base_ref }}' == 'experimental' ]]; then
7575
# Pick Agda version for experimental
76-
echo "AGDA_COMMIT=tags/v2.6.1.3.20210524" >> $GITHUB_ENV;
76+
echo "AGDA_COMMIT=tags/v2.6.2" >> $GITHUB_ENV;
7777
echo "AGDA_HTML_DIR=html/experimental" >> $GITHUB_ENV
7878
fi
7979
@@ -130,7 +130,7 @@ jobs:
130130
cd ..
131131
132132
########################################################################
133-
## TESTING AND DEPLOYMENT
133+
## TESTING
134134
########################################################################
135135

136136
# By default github actions do not pull the repo
@@ -139,13 +139,22 @@ jobs:
139139

140140
- name: Test stdlib
141141
run: |
142-
${{ env.CABAL_INSTALL }} agda-stdlib-utils.cabal
143142
cabal run GenerateEverything
144143
cp travis/* .
145144
./index.sh
146145
${{ env.AGDA }} --safe EverythingSafe.agda
147146
${{ env.AGDA }} index.agda
148147
148+
- name: Golden testing
149+
run: |
150+
${{ env.CABAL_INSTALL }} --lib clock
151+
make testsuite INTERACTIVE='' AGDA_EXEC='~/.cabal/bin/agda'
152+
153+
154+
########################################################################
155+
## DOC DEPLOYMENT
156+
########################################################################
157+
149158
# We start by retrieving the currently deployed docs
150159
# We remove the content that is in the directory we are going to populate
151160
# so that stale files corresponding to deleted modules do not accumulate.
@@ -157,25 +166,11 @@ jobs:
157166
rm -f '${{ env.AGDA_HTML_DIR }}'/*.css
158167
${{ env.AGDA }} --html --html-dir ${{ env.AGDA_HTML_DIR }} index.agda
159168
160-
# This is a massive hack at the moment
161-
# - name: Compile stdlib
162-
# run: |
163-
# ${{ env.AGDA }} -c --no-main --ghc-dont-call-ghc --compile-dir=tmp Everything.agda
164-
# cd tmp
165-
# yes | cabal init --interactive
166-
# head -n -17 tmp.cabal > tmp
167-
# mv tmp tmp.cabal
168-
# cat ../travis/ghc-options >> tmp.cabal
169-
# cabal build
170-
171-
## ${{ env.AGDA }} -c README/Foreign/Haskell.agda && ./Haskell
172-
173-
174169
- name: Deploy HTML
175170
uses: JamesIves/[email protected]
176171
if: ${{ success() && env.AGDA_DEPLOY }}
177172

178173
with:
179174
branch: gh-pages
180175
folder: html
181-
git-config-name: Github Actions
176+
git-config-name: Github Actions

.gitignore

+4
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,11 @@ Everything.agda
1919
EverythingSafe.agda
2020
EverythingSafeGuardedness.agda
2121
EverythingSafeSizedTypes.agda
22+
failures
2223
GenerateEverything
2324
Haskell
2425
html
26+
log
2527
MAlonzo
28+
output
29+
runtests

CHANGELOG.md

+100
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,53 @@ The library has been tested using Agda 2.6.2.
66
Highlights
77
----------
88

9+
* A golden testing library in `Test.Golden`. This allows you to run a set
10+
of tests and make sure their output matches an expected `golden' value.
11+
The test runner has many options: filtering tests by name, dumping the
12+
list of failures to a file, timing the runs, coloured output, etc.
13+
Cf. the comments in `Test.Golden` and the standard library's own tests
14+
in `tests/` for documentation on how to use the library.
15+
916
Bug-fixes
1017
---------
1118

19+
* In `System.Exit`, the `ExitFailure` constructor is now carrying an integer
20+
rather than a natural. The previous binding was incorrectly assuming that
21+
all exit codes where non-negative.
22+
1223
Non-backwards compatible changes
1324
--------------------------------
1425

26+
* In `Text.Pretty`, `Doc` is now a record rather than a type alias. This
27+
helps Agda reconstruct the `width` parameter when the module is opened
28+
without it being applied. In particular this allows users to write
29+
width-generic pretty printers and only pick a width when calling the
30+
renderer by using this import style:
31+
```
32+
open import Text.Pretty using (Doc; render)
33+
-- ^-- no width parameter for Doc & render
34+
open module Pretty {w} = Text.Pretty w hiding (Doc; render)
35+
-- ^-- implicit width parameter for the combinators
36+
37+
pretty : Doc w
38+
pretty = ? -- you can use the combinators here and there won't be any
39+
-- issues related to the fact that `w` cannot be reconstructed
40+
-- anymore
41+
42+
main = do
43+
-- you can now use the same pretty with different widths:
44+
putStrLn $ render 40 pretty
45+
putStrLn $ render 80 pretty
46+
```
47+
48+
* In `Text.Regex.Search` the `searchND` function finding infix matches has
49+
been tweaked to make sure the returned solution is a local maximum in terms
50+
of length. It used to be that we would make the match start as late as
51+
possible and finish as early as possible. It's now the reverse.
52+
53+
So `[a-zA-Z]+.agdai?` run on "the path _build/Main.agdai corresponds to"
54+
will return "Main.agdai" when it used to be happy to just return "n.agda".
55+
1556
Deprecated modules
1657
------------------
1758

@@ -27,6 +68,29 @@ New modules
2768
Algebra.Morphism.Construct.Identity
2869
```
2970

71+
* Show module for unnormalised rationals:
72+
```
73+
Data.Rational.Unnormalised.Show
74+
```
75+
76+
* Various system types and primitives:
77+
```
78+
System.Clock.Primitive
79+
System.Clock
80+
System.Console.ANSI
81+
System.Directory.Primitive
82+
System.Directory
83+
System.FilePath.Posix.Primitive
84+
System.FilePath.Posix
85+
System.Process.Primitive
86+
System.Process
87+
```
88+
89+
* A golden testing library with test pools, an options parser, a runner:
90+
```
91+
Test.Golden
92+
```
93+
3094
Other minor additions
3195
---------------------
3296

@@ -53,3 +117,39 @@ Other minor additions
53117
∧-isCommutativeSemigroup : IsCommutativeSemigroup ∧
54118
```
55119
and their corresponding algebraic substructures.
120+
121+
* In `Data.String.Properties`:
122+
```
123+
≤-isDecTotalOrder-≈ : IsDecTotalOrder _≈_ _≤_
124+
≤-decTotalOrder-≈ : DecTotalOrder _ _ _
125+
```
126+
127+
* In `IO`:
128+
```
129+
Colist.forM : Colist A → (A → IO B) → IO (Colist B)
130+
Colist.forM′ : Colist A → (A → IO B) → IO ⊤
131+
132+
List.forM : List A → (A → IO B) → IO (List B)
133+
List.forM′ : List A → (A → IO B) → IO ⊤
134+
```
135+
136+
* In `IO.Base`:
137+
```
138+
lift! : IO A → IO (Lift b A)
139+
_<$_ : B → IO A → IO B
140+
_=<<_ : (A → IO B) → IO A → IO B
141+
_<<_ : IO B → IO A → IO B
142+
lift′ : Prim.IO ⊤ → IO {a} ⊤
143+
144+
when : Bool → IO {a} ⊤ → IO ⊤
145+
unless : Bool → IO {a} ⊤ → IO ⊤
146+
147+
whenJust : Maybe A → (A → IO {a} ⊤) → IO ⊤
148+
untilJust : IO (Maybe A) → IO A
149+
```
150+
151+
* In `System.Exit`:
152+
```
153+
isSuccess : ExitCode → Bool
154+
isFailure : ExitCode → Bool
155+
```

GNUmakefile

+3
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ AGDA=$(AGDA_EXEC) $(RTS_OPTIONS)
1010
test: Everything.agda check-whitespace
1111
$(AGDA) -i. -isrc README.agda
1212

13+
testsuite:
14+
$(MAKE) -C tests test AGDA="$(AGDA)" AGDA_EXEC="$(AGDA_EXEC)" only=$(only)
15+
1316
check-whitespace:
1417
cabal exec -- fix-whitespace --check
1518

GenerateEverything.hs

+9
Original file line numberDiff line numberDiff line change
@@ -56,10 +56,19 @@ unsafeModules = map modToFile
5656
, "IO.Primitive.Infinite"
5757
, "IO.Primitive.Finite"
5858
, "Relation.Binary.PropositionalEquality.TrustMe"
59+
, "System.Clock"
60+
, "System.Clock.Primitive"
61+
, "System.Directory"
62+
, "System.Directory.Primitive"
5963
, "System.Environment"
6064
, "System.Environment.Primitive"
6165
, "System.Exit"
6266
, "System.Exit.Primitive"
67+
, "System.FilePath.Posix"
68+
, "System.FilePath.Posix.Primitive"
69+
, "System.Process"
70+
, "System.Process.Primitive"
71+
, "Test.Golden"
6372
, "Text.Pretty.Core"
6473
, "Text.Pretty"
6574
] ++ sizedTypesModules
+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
------------------------------------------------------------------------
2+
-- The Agda standard library
3+
--
4+
-- Showing unnormalised rational numbers
5+
------------------------------------------------------------------------
6+
7+
{-# OPTIONS --without-K --safe #-}
8+
9+
module Data.Rational.Unnormalised.Show where
10+
11+
import Data.Integer.Show as ℤ
12+
open import Data.Rational.Unnormalised.Base
13+
open import Data.String.Base using (String; _++_)
14+
15+
show : ℚᵘ String
16+
show p = ℤ.show (↥ p) ++ "/" ++ ℤ.show (↧ p)

src/Data/String/Properties.agda

+12
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,18 @@ x <? y = StrictLex.<-decidable Charₚ._≟_ Charₚ._<?_ (toList x) (toList y)
129129
toList
130130
(StrictLex.≤-isDecPartialOrder Charₚ.<-isStrictTotalOrder)
131131

132+
≤-isDecTotalOrder-≈ : IsDecTotalOrder _≈_ _≤_
133+
≤-isDecTotalOrder-≈ =
134+
On.isDecTotalOrder
135+
toList
136+
(StrictLex.≤-isDecTotalOrder Charₚ.<-isStrictTotalOrder)
137+
138+
≤-decTotalOrder-≈ : DecTotalOrder _ _ _
139+
≤-decTotalOrder-≈ =
140+
On.decTotalOrder
141+
(StrictLex.≤-decTotalOrder Charₚ.<-strictTotalOrder)
142+
toList
143+
132144
≤-decPoset-≈ : DecPoset _ _ _
133145
≤-decPoset-≈ =
134146
On.decPoset

src/IO.agda

+13-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ open import Codata.Musical.Costring
1313
open import Data.Unit.Polymorphic.Base
1414
open import Data.String.Base
1515
import Data.Unit.Base as Unit0
16-
open import Function.Base using (_∘_)
16+
open import Function.Base using (_∘_; flip)
1717
import IO.Primitive as Prim
1818
open import Level
1919

@@ -54,6 +54,12 @@ module Colist where
5454
mapM′ : (A IO B) Colist A IO ⊤
5555
mapM′ f = sequence′ ∘ map f
5656

57+
forM : Colist A (A IO B) IO (Colist B)
58+
forM = flip mapM
59+
60+
forM′ : Colist A (A IO B) IO ⊤
61+
forM′ = flip mapM′
62+
5763
module List where
5864

5965
open import Data.List.Base
@@ -75,6 +81,12 @@ module List where
7581
mapM′ : (A IO B) List A IO ⊤
7682
mapM′ f = sequence′ ∘ map f
7783

84+
forM : List A (A IO B) IO (List B)
85+
forM = flip mapM
86+
87+
forM′ : List A (A IO B) IO ⊤
88+
forM′ = flip mapM′
89+
7890
------------------------------------------------------------------------
7991
-- Simple lazy IO
8092

0 commit comments

Comments
 (0)