Skip to content

Commit 1ccf535

Browse files
committed
Update conformance-tests-sonobuoy.md
Signed-off-by: Toni Finger <[email protected]>
1 parent c9eed98 commit 1ccf535

File tree

1 file changed

+42
-14
lines changed

1 file changed

+42
-14
lines changed

contributor-docs/development/tests/conformance-tests-sonobuoy.md

+42-14
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ This requires a user to write the conformance tests in Golang, as this is the la
2323
* [docker][docker-installation]
2424
* [kind][kind-installation]
2525
* [sonobuoy][sonobuoy-installation]
26+
* [yq][yq-installation]
27+
* [jq][jq-installation]
2628

2729
```bash
2830
go install github.com/vmware-tanzu/sonobuoy@latest
@@ -34,7 +36,7 @@ go install github.com/vmware-tanzu/sonobuoy@latest
3436

3537
```bash
3638
git clone https://github.com/SovereignCloudStack/standards
37-
cd standards/Tests/kaas/kaas-sonobuoy-go-example-e2e-framework/
39+
cd standards/Tests/kaas/kaas-sonobuoy-tests/
3840
```
3941

4042
#### 2. Check the prerequisites
@@ -54,16 +56,12 @@ Once all the prerequisite software is installed, you can proceed by starting an
5456
make dev-setup
5557
```
5658

57-
#### 4. Setting environment variables for the image build process
59+
#### 4. Setting environment variables for the development process
5860

59-
```bash
60-
kubectl config view
61-
```
61+
Set the number code of the standard you are currently working on.
6262

6363
```bash
64-
export IMAGE_VERSION_TAG="dev"
65-
export K8S_HOST=<kind-cluster-ip>
66-
export K8S_PORT=<kind-cluster-port>
64+
export TESTFUNCTION_CODE=<number code of the standard>
6765
```
6866

6967
### Create a test
@@ -81,14 +79,15 @@ Pretend that the fictitious standard here stipulates that at least one pod MUST
8179
The `scs_k8s_tests` directory contains the Golang files that define the tests.
8280

8381
```bash
84-
cd standards/Tests/kaas/kaas-sonobuoy-go-example-e2e-framework/scs_k8s_tests
82+
cd standards/Tests/kaas/kaas-sonobuoy-tests/scs_k8s_tests
8583
```
8684

8785
First create a test file according to your standard and adhere to the naming convention accordingly.
8886

8987
* The prefix MUST contain the name of the standard in lower case letters.
9088
* As a suffix, the file must end with "_test.go".
9189

90+
> [!NOTE]
9291
> The suffix requirement comes from the go test framework itself. All test files must end with `_test.go`.
9392
> Otherwise they will not be selected by the test environment.
9493
@@ -103,10 +102,13 @@ As an example, we test here whether there are more than zero pods in the namespa
103102
The execution of this test should fail by default as there should be no pods in the namespace and the namespace itself should not exist.
104103
The aim is to display the results of a failed test so that we can show their interpretation in a later step.
105104

105+
> [!NOTE]
106106
> Attention!!!: in order for the framework to select the functions for testing, their names must begin with "TEST_" in accordance with the naming convention of the golang test framework.
107-
> TODO:!!! link to golang test framework docs
107+
> The framework in use is [kubernetes-sigs/e2e-framework][e2e-framework].
108+
> They also provide examples. Before you start implementing your own tests, you should check whether you can use one of the examples as a starting point for your own implementation.
109+
> Have a look at: [kubernetes-sigs/e2e-framework/examples][e2e-framework-examples]
108110
109-
Copy the following text into the file `scs_0299_v1_example_standard_test.go`:
111+
As an example of this step-by-step guide, copy the following text into the file `scs_0299_v1_example_standard_test.go`:
110112

111113
```go
112114
package scs_k8s_tests
@@ -147,7 +149,7 @@ func Test_scs_0299_TestListPodsFailing(t *testing.T) {
147149

148150
```
149151

150-
#### 3. Build the test image, upload it to the art cluster and run it
152+
#### 3. Build the test image, upload it to the kind cluster and run it
151153

152154
To create the image, execute the following:
153155

@@ -174,8 +176,8 @@ sonobuoy status
174176

175177
```bash
176178
make dev-result
177-
cat results/plugins/scsconformance/results/global/out.json
178-
cat results/plugins/scsconformance/sonobuoy_results.yaml
179+
cat results/plugins/scsconformance/results/global/out.json | jq
180+
cat results/plugins/scsconformance/sonobuoy_results.yaml | yq
179181
```
180182

181183
Once all tests have been executed successfully, you can read the results and receive feedback.
@@ -193,10 +195,36 @@ make dev-clean
193195
make dev-purge
194196
```
195197

198+
### (optional) Run all at once:
199+
200+
In addition, you can trigger all processes from above with one command. There is a single target in the Makefile for this:
201+
202+
```bash
203+
make dev-rerun
204+
```
205+
206+
### (optional) Run only your testfunctions
207+
208+
The above process always executes all tests. This can take some time.
209+
Therefore, it is also possible to run only your standard test functions outside the scope of sonobuoy:
210+
211+
```bash
212+
export TESTFUNCTION_CODE="0299"
213+
make test-function
214+
```
215+
216+
> [!NOTE]
217+
> [e2e-framework-examples]
218+
196219
[sonobuoy]: https://sonobuoy.io/
197220
[sonbouy-decision-record]: https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0200-v1-using-sonobuoy-for-kaas-conformance-tests.md
198221
[k8s-conformance]: https://github.com/cncf/k8s-conformance/blob/master/instructions.md
199222
[docker-installation]: https://docs.docker.com/engine/install/
200223
[sonobuoy-installation]: https://sonobuoy.io/docs/v0.57.1/#installation
201224
[kind-installation]: https://kind.sigs.k8s.io/docs/user/quick-start/#installation
202225
[scs-sonobuoy-example-guide]: https://github.com/SovereignCloudStack/standards/tree/main/Tests/kaas/kaas-sonobuoy-go-example-e2e-framework#sonobuoy-usage-for-development-of-tests
226+
227+
[yq-installation]:https://github.com/mikefarah/yq/?tab=readme-ov-file#install
228+
[jq-installation]:https://jqlang.github.io/jq/download/
229+
[e2e-framework]:https://github.com/kubernetes-sigs/e2e-framework
230+
[e2e-framework-examples]:https://github.com/kubernetes-sigs/e2e-framework/tree/main/examples

0 commit comments

Comments
 (0)