Skip to content

Adds support for using Terraform modules cached locally #940

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

Rchanger
Copy link
Contributor

@Rchanger Rchanger commented Jul 22, 2021

Add: support for using the terraform init cache for remote modules rather than downloading it again. It will try to use the remote module from terraform cache if --use-terraform-cache flag is set and if the module is not found in cache it will be downloaded

Usage

   terrascan scan -d testdata/terraform_cache_use_in_scan -i terraform --use-terraform-cache  

Closes #936

@codecov
Copy link

codecov bot commented Jul 22, 2021

Codecov Report

Merging #940 (5cfe25c) into master (11db729) will decrease coverage by 0.16%.
The diff coverage is 66.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #940      +/-   ##
==========================================
- Coverage   79.10%   78.93%   -0.17%     
==========================================
  Files         220      221       +1     
  Lines        5359     5413      +54     
==========================================
+ Hits         4239     4273      +34     
- Misses        866      880      +14     
- Partials      254      260       +6     
Impacted Files Coverage Δ
pkg/iac-providers/arm/v1/load-dir.go 67.27% <0.00%> (ø)
pkg/iac-providers/arm/v1/load-file.go 70.83% <ø> (ø)
pkg/iac-providers/cft/v1/load-file.go 80.76% <ø> (ø)
pkg/iac-providers/docker/v1/load-file.go 92.85% <ø> (ø)
pkg/iac-providers/helm/v3/load-dir.go 82.89% <ø> (ø)
pkg/iac-providers/helm/v3/load-file.go 100.00% <ø> (ø)
pkg/iac-providers/kubernetes/v1/load-dir.go 72.72% <0.00%> (ø)
pkg/iac-providers/kubernetes/v1/load-file.go 71.87% <ø> (ø)
pkg/iac-providers/kustomize/v3/load-dir.go 71.69% <ø> (ø)
pkg/iac-providers/kustomize/v3/load-file.go 100.00% <ø> (ø)
... and 19 more

@Rchanger Rchanger force-pushed the sr_support_for_using_tf_cache branch from ec41505 to 6e79069 Compare July 22, 2021 17:01
@Rchanger Rchanger marked this pull request as ready for review July 23, 2021 13:00
@Rchanger Rchanger linked an issue Jul 23, 2021 that may be closed by this pull request
Fixes typo
Rchanger and others added 9 commits July 24, 2021 13:38
- changed interface method signature to accept map parameter so that we can pass any no of flag to    implementer without changing method signature.
- moved the non recursive flag to map to achieve uniformity
- In terraform used switch for flag identification so that in future new flag or option provided just need to add another case rather than if/else
updated all links .
We doSupport TLS certificate/key in existing secrets
Adds pre-commit integration for Terrascan

Co-authored-by: mihirhasan <[email protected]>
@cesar-rodriguez cesar-rodriguez changed the title add: support for using already downloaded remote module add: support for using already downloaded Terraform module Aug 2, 2021
Copy link
Contributor

@cesar-rodriguez cesar-rodriguez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to test this feature, but seems like it's not deducting the module directory correctly. I'm using Terraform v0.15.5 with this main.tf file:

module "vault" {
  source  = "hashicorp/vault/aws"
  version = "0.16.0"
  ssh_key_name     = ""
  create_dns_entry = false
}

When I run:

terrascan scan -i terraform --use-terraform-cache -l debug

I get this error message and it also downloads the modules instead of using the local cache

2021-08-01T22:52:56.103-0400    error   utils/dir.go:49 directory /Users/myusername/programming/vault/.terraform/modules/vault/.terraform/modules does not exist.

2021-08-01T22:52:56.103-0400    debug   downloader/getter.go:49 downloading "github.com/hashicorp/terraform-aws-consul.git?ref=v0.8.0" to "/var/folders/2g/9lkfm6ld2lv350svwr15fdgc0000gn/T/n1ukaa"

The cached modules are actually located at:

/Users/myusername/programming/vault/.terraform/modules

- modified code to make scan directory non recursively when --use-terraform-cache is used.
- modified scan command flag description and docs accordingly.
cesar-rodriguez
cesar-rodriguez previously approved these changes Aug 4, 2021
@cesar-rodriguez cesar-rodriguez changed the base branch from master to temp August 4, 2021 02:55
@cesar-rodriguez cesar-rodriguez changed the base branch from temp to master August 4, 2021 02:55
@cesar-rodriguez cesar-rodriguez dismissed their stale review August 4, 2021 02:55

The base branch was changed.

cesar-rodriguez
cesar-rodriguez previously approved these changes Aug 4, 2021
cesar-rodriguez
cesar-rodriguez previously approved these changes Aug 4, 2021
@cesar-rodriguez cesar-rodriguez changed the title add: support for using already downloaded Terraform module Adds support for using already Terraform modules cached locally Aug 4, 2021
@cesar-rodriguez cesar-rodriguez changed the title Adds support for using already Terraform modules cached locally Adds support for using Terraform modules cached locally Aug 4, 2021
- modified docker LoadIacDir and LoadIacFile method to accept map parameter
- registered docker as cloud provider so that docker policies will be used from '.terrascan' folder
@sonarqubecloud
Copy link

sonarqubecloud bot commented Aug 4, 2021

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ensure remote modules are downloaded only once
7 participants