Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable test coverage + upload results in workflow #91

Merged
merged 3 commits into from
Jun 1, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,9 @@ jobs:

- name: Run Go Tests
run: make test

- name: Upload Test Coverage results
uses: actions/upload-artifact@v2
with:
name: lib-test-coverage-html
path: tests/v2/lib-test-coverage.html
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ main

# File created running tests
tests/**/tmp/
tests/v2/lib-test-coverage.*


# Mac related files
.DS_Store
Expand Down
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ bin:

.PHONY: test
test:
go test ./...
go test -coverprofile tests/v2/lib-test-coverage.out -v ./...
go tool cover -html=tests/v2/lib-test-coverage.out -o tests/v2/lib-test-coverage.html

.PHONY: clean
clean:
Expand Down
25 changes: 17 additions & 8 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## About

The tests use the go language and are intended to test every apsect of the parser for every schema attribute. Some basic aspects of the tests:
The tests use the go language and are intended to test every aspect of the parser for every schema attribute. Some basic aspects of the tests:

* A first test (parser_v200_schema_test.go) feeds pre-created devfiles to the parser to ensure the parser can parse all attribues and return an appropriate error when the devfile contains an error. This test is not currently available.
* A second set of tests (parser_v200_verify_test.go) create devfile content at runtime:
Expand Down Expand Up @@ -35,23 +35,32 @@ The tests which generate devfiles with random content at run time currently cove
## Test structure

* From this repository
- `test/v2/libraryTest/library-test.go`: The go unit test program
- `test/v2/utils/library/*-utils.go` : utilites, used by the test, which contain functions uniqiue to the library tests. Mostly contain the code to modify and check devfile content.
- `tests/v2/libraryTest/library-test.go`: The go unit test program
- `tests/v2/utils/library/*-utils.go` : utilites, used by the test, which contain functions uniqiue to the library tests. Mostly contain the code to modify and check devfile content.
* From the [api respository](https://github.com/devfile/api/tree/master/test/v200/utils/common)
- `test/v200/utils/common/*-utils.go` : utilites, used by the test, which are also used by the api tests. Mostly contain the code to generate valid devfile content.
- `tests/v200/utils/common/*-utils.go` : utilites, used by the test, which are also used by the api tests. Mostly contain the code to generate valid devfile content.

## Running the tests
## Running the tests locally

1. Go to directory /tests/v2/libraryTest
1. Run ```go test``` or ```go test -v```
1. Go to the ```/library``` directory
1. Run ```Make test```
1. The test creates the following files:
1. ```./tmp/test.log``` contains log output from the tests.
1. ```./tmp/library_test/Test_*.yaml``` are the devfiles which are randomly generated at runtime. The file name matches the name of the test function which resulted in them being created.
1. If a test detects an error when comparing properties returned by the parser with expected properties
* ```./tmp/library_test/Test_*_<property id>_Parser.yaml``` - property as returned by the parser
* ```./tmp/library_test/Test_*_<property id>_Test.yaml``` - property as expected by the test
1. ```tests/v2/lib-test-coverage.html``` which is the test coverage report for the test run. You can open this file up in a browser and inspect the results to determine the gaps you may have in testing

Note: each run of the test removes the existing contents of the ```./tmp``` directory

## Viewing test results from a workflow

The tests run automatically with every PR or Push action. You can see the results in the `devfile/library` repo's `Actions` view:

Note: each run of the test removes the existing conents of the ```./tmp``` directory
1. Select the `Validate PRs` workflow and click on the PR you're interested in
1. To view the console output, select the `Build` job and expand the `Run Go Tests` step. This will give you a summary of the tests that were executed and their respective status
1. To view the test coverage report, click on the `Summary` page and you should see an `Artifacts` section with the `lib-test-coverage-html` file available for download.