Refactor HCP Vault TF acceptance tests for Azure #441

codergs · 2023-01-17T15:19:59Z

This PR adds acceptance tests with Azure cloud provider. Not all features are available under Azure cloud.

refactor is done so that we can use same code base for AWS And Azure cloud
separated out perf replication tests to a separate file - no refactor is done for this at the moment.
fixed a regression on perf replication test suites

🛠️ Description

🏗️ Acceptance tests

Are there any feature flags that are required to use this functionality?
Have you added an acceptance test for the functionality being added?
Have you run the acceptance tests on this branch?

Output from acceptance testing:

Wed Oct 12 16:48:43 EDT 2022 hashicorp/terraform-provider-hcp-internal - (codergs-HVT-2273) > TF_LOG_PATH=out.text TF_LOG=TRACE TF_ACC=1 go test ./internal/... -parallel 20 -v -run=TestAccVaultClusterAWS -timeout 210m     
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-hcp/internal/clients    (cached) [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-hcp/internal/consul     (cached) [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-hcp/internal/input      (cached) [no tests to run]
=== RUN   TestAccVaultClusterAWS
=== PAUSE TestAccVaultClusterAWS
=== CONT  TestAccVaultClusterAWS
--- PASS: TestAccVaultClusterAWS (2630.25s)
PASS
ok      github.com/hashicorp/terraform-provider-hcp/internal/provider   2630.911s

internal/provider/resource_vault_cluster_perf_replication_test.go

internal/provider/resource_vault_cluster_test.go

internal/provider/resource_vault_cluster_const_test.go

hashicorp-cla · 2023-01-19T18:36:14Z

All committers have signed the CLA.

aoripov · 2023-01-19T20:11:41Z

@bcmdarroch I ran the tests locally:

~/projects/terraform-provider-hcp (codergs-HVT-2273)$ TF_LOG_PATH=out.text TF_LOG=TRACE TF_ACC=1 go test ./internal/... -parallel 20 -v -run=TestAccVaultClusterAWS -timeout 210m
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/clients	(cached) [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/consul	(cached) [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/input	(cached) [no tests to run]
=== RUN   TestAccVaultClusterAWS
=== PAUSE TestAccVaultClusterAWS
=== CONT  TestAccVaultClusterAWS
--- PASS: TestAccVaultClusterAWS (4451.42s)
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/provider	4452.169s

codergs · 2023-01-19T20:26:24Z

@aoripov @bcmdarroch can I please trouble you with azure tests too?

internal/provider/resource_vault_cluster_test.go

internal/provider/resource_vault_cluster_perf_replication_test.go

aoripov · 2023-01-25T13:09:49Z

Azure tests:

TF_LOG_PATH=out.main.text TF_LOG=TRACE TF_ACC=1 go test ./internal/... -parallel 20 -v -run=TestAccVaultClusterAzure -timeout 240m
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/clients	(cached) [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/consul	(cached) [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/input	(cached) [no tests to run]
=== RUN   TestAccVaultClusterAzure
=== PAUSE TestAccVaultClusterAzure
=== CONT  TestAccVaultClusterAzure
--- PASS: TestAccVaultClusterAzure (4493.64s)
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/provider	4494.746s

aoripov · 2023-01-25T15:07:16Z

AWS tests.
Ignore failure. It failed on the last step of deleting an HVN because of an increased cadence latency.

TF_LOG_PATH=out.main.text TF_LOG=TRACE TF_ACC=1 go test ./internal/... -parallel 20 -v -run=TestAccVaultClusterAWS -timeout 240m
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/clients	(cached) [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/consul	(cached) [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/input	(cached) [no tests to run]
=== RUN   TestAccVaultClusterAWS
=== PAUSE TestAccVaultClusterAWS
=== CONT  TestAccVaultClusterAWS
    testing_new.go:84: Error running post-test destroy, there may be dangling resources: exit status 1

        Error: unable to delete HVN (test-hvn-aws-202301251324): context canceled waiting for delete HVN operation (5a81e5f1-b4b8-4a46-b801-8ef0fdb42c8b) to complete

--- FAIL: TestAccVaultClusterAWS (7092.17s)
FAIL
FAIL	github.com/hashicorp/terraform-provider-hcp/internal/provider	7092.827s
FAIL

aoripov · 2023-01-25T17:21:25Z

Performance replication tests. Same as with AWS tests they failed on the very last step - HVN deletion.

TF_LOG_PATH=repl.1.text TF_LOG=TRACE TF_ACC=1 go test ./internal/... -parallel 20 -v -run=TestAccPerformanceReplication_ValidationsAws -timeout 240m
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/clients	(cached) [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/consul	(cached) [no tests to run]
testing: warning: no tests to run
PASS
ok  	github.com/hashicorp/terraform-provider-hcp/internal/input	(cached) [no tests to run]
=== RUN   TestAccPerformanceReplication_ValidationsAws
    testing_new.go:84: Error running post-test destroy, there may be dangling resources: exit status 1

        Error: unable to delete HVN (test-perf-hvn-2-202301251411): context canceled waiting for delete HVN operation (43daf7c0-f1bf-4e0f-a4d4-43bd64e6b9b5) to complete


        Error: unable to delete HVN (test-perf-hvn-1-202301251411): context canceled waiting for delete HVN operation (0e0d2f89-c6d1-4e80-bab5-b594a387f9f6) to complete

--- FAIL: TestAccPerformanceReplication_ValidationsAws (12984.60s)
FAIL
FAIL	github.com/hashicorp/terraform-provider-hcp/internal/provider	12985.886s
FAIL

codergs

LGTM, thanks for the testing.

bcmdarroch

Did one final test and all the Vault tests passed. Thanks for fixing those IDs!

codergs requested review from a team, helenfufu, biazmoreira and himran92 and removed request for biazmoreira and helenfufu January 17, 2023 15:19

bcmdarroch reviewed Jan 18, 2023

View reviewed changes

internal/provider/resource_vault_cluster_perf_replication_test.go Outdated Show resolved Hide resolved

bcmdarroch reviewed Jan 18, 2023

View reviewed changes

internal/provider/resource_vault_cluster_test.go Outdated Show resolved Hide resolved

aoripov self-requested a review January 19, 2023 15:22

bcmdarroch reviewed Jan 19, 2023

View reviewed changes

internal/provider/resource_vault_cluster_const_test.go Outdated Show resolved Hide resolved

bcmdarroch reviewed Jan 19, 2023

View reviewed changes

internal/provider/resource_vault_cluster_test.go Outdated Show resolved Hide resolved

bcmdarroch reviewed Jan 19, 2023

View reviewed changes

internal/provider/resource_vault_cluster_perf_replication_test.go Outdated Show resolved Hide resolved

aoripov requested a review from bcmdarroch January 24, 2023 18:00

codergs commented Jan 25, 2023

View reviewed changes

bcmdarroch approved these changes Jan 26, 2023

View reviewed changes

codergs and others added 8 commits January 26, 2023 11:12

Refactor HCP Vault TF acceptance tests for Azure

5dc1679

style check

ba1a24c

change logs

0cd7e63

typo update

066eed0

Change makefile to run tests in parallel

6184cbc

Make resource ids unique

8d67e75

Fix typo

d9beb81

Refactor perf replication tests and shorten resource ids

f088cc8

aoripov added 2 commits January 26, 2023 11:12

Remove old perf replication tests

956279b

Increase timeouts for create/update/delete

3ffd53f

bcmdarroch force-pushed the codergs-HVT-2273 branch from 83d5bba to 3ffd53f Compare January 26, 2023 16:12

bcmdarroch merged commit ebb8440 into main Jan 26, 2023

bcmdarroch deleted the codergs-HVT-2273 branch January 26, 2023 22:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor HCP Vault TF acceptance tests for Azure #441

Refactor HCP Vault TF acceptance tests for Azure #441

codergs commented Jan 17, 2023 •

edited

Loading

hashicorp-cla commented Jan 19, 2023 •

edited

Loading

aoripov commented Jan 19, 2023

codergs commented Jan 19, 2023

aoripov commented Jan 25, 2023

aoripov commented Jan 25, 2023

aoripov commented Jan 25, 2023

codergs left a comment

bcmdarroch left a comment

Refactor HCP Vault TF acceptance tests for Azure #441

Refactor HCP Vault TF acceptance tests for Azure #441

Conversation

codergs commented Jan 17, 2023 • edited Loading

🛠️ Description

🏗️ Acceptance tests

hashicorp-cla commented Jan 19, 2023 • edited Loading

aoripov commented Jan 19, 2023

codergs commented Jan 19, 2023

aoripov commented Jan 25, 2023

aoripov commented Jan 25, 2023

aoripov commented Jan 25, 2023

codergs left a comment

Choose a reason for hiding this comment

bcmdarroch left a comment

Choose a reason for hiding this comment

codergs commented Jan 17, 2023 •

edited

Loading

hashicorp-cla commented Jan 19, 2023 •

edited

Loading