Enable test coverage + upload results in workflow (#476)

* enable test coverage + upload results in workflow

* Update Readme

* create a separate step for  report generation

* fix path

Author: kim-tsao

.github/workflows/ci.yaml

@@ -32,7 +32,17 @@ jobs:
       run: ./docker-run.sh ./validate-samples.sh
 
     - name: Run GO tests
-      run: go test -v ./...
+      run: go test -coverprofile test/v200/api-test-coverage.out -v ./...
+
+    - name: Generate test coverage report
+      run: go tool cover -html=test/v200/api-test-coverage.out -o test/v200/api-test-coverage.html
+      
+    - name: Upload Test Coverage results
+      uses: actions/upload-artifact@v2
+      with:
+        name: api-test-coverage-html
+        path: test/v200/api-test-coverage.html
+
 
     - name: Check GO mod state      
       run: |

.gitignore

@@ -3,6 +3,7 @@ build/_output
 build/_test
 test/**/tmp
 test/go/pkg
+test/v200/api-test-coverage.*
 # Created by https://www.gitignore.io/api/go,vim,emacs,visualstudiocode
 ### Emacs ###
 # -*- mode: gitignore; -*-

test/README.md

@@ -16,9 +16,9 @@ The API tests are intended to provide a comprehensive verification of the devfil
     - `test-xxxxxxx.json` : these files are the top level json files, they define the schema to verify and the test files to run.
     - `xxxxxx-tests.json` : these are the test files which contain individual tests which provide the yaml snippets to combine and the expected result.
 
-## Running tests
+## Running tests locally
 
-from the test/go/src/test directory run 
+From the `test/v200/schemaTest` directory run 
 - `go test -v`
 
 The test will read each of the test-xxxxxx.json files and run the tests defined within. The generated .yaml files used for the tests are created in a `tmp/test-xxxxxx/` directory. These files are not deleted when the test finishes so they can be used to assess any errors, however they will be deleted by a subsequent run of the test. Running the test with the -v option ensures you see a full list of passes and failures. 
@@ -57,14 +57,26 @@ A new test approach, shared with the library repository for testing valid devfil
 - `test/v200/utils/common/*-utils.go` : utilites, used by the test, which are also used by the library tests. Mostly contain the code to generate valid devfile content.
 
 
-## Running tests
+## Running tests locally
 
 from the `test/v200/apiTest/` directory run
 - `go test -v`
 
-* The test will generate a set of valid devfile.yaml files in `test/v200/apiTest/tmp/api-test/
+* The test will generate a set of valid devfile.yaml files in `test/v200/apiTest/tmp/api-test/`
 * The test will generate a log file:  `test/v200/apiTest/tmp/test.log`
 * Each run of the test removes the  `test/v200/apiTest/tmp` directory from the previous run.
 
+# Run test coverage analysis and reporting locally
 
+This is useful to determine if there are any gaps in testing.  Run these steps at the root directory `/api` to get a report of all the overall tests, including the ones for api and schema
 
+- `go test -coverprofile test/v200/api-test-coverage.out -v ./...`
+- `go tool cover -html=test/v200/api-test-coverage.out -o test/v200/api-test-coverage.html`
+
+## Viewing test results from a workflow
+
+The tests run automatically with every PR or Push action.  You can see the results in the `devfile/api` repo's `Actions` view:
+
+1.  Select the `CI` workflow and click on the PR you're interested in
+1.  To view the console output, select the `build-and-validate` job and expand the `Run Go Tests` step.  This will give you a summary of the tests that were executed and their respective status 
+1.  To view the test coverage report, click on the `Summary` page and you should see an `Artifacts` section with the `api-test-coverage-html` file available for download.