Cycling Dependencies with eks addons and managed node groups with 18.4.1

[solidnerd]

Description

During the upgrade of 18.4.1 (#1840) there was a fixed made to fix the ordering problem of eks addons and node groups. This fix currently dont work since its introduces in terraform a cycling dependencie.

Versions

• Terraform:

tf -version
Terraform v1.1.4
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v3.74.0
+ provider registry.terraform.io/hashicorp/cloudinit v2.2.0
+ provider registry.terraform.io/hashicorp/kubernetes v2.7.1
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/tls v3.1.0

Your version of Terraform is out of date! The latest version
is 1.1.5. You can update by downloading from https://www.terraform.io/downloads.html

• Provider(s):

tf providers -version
Terraform v1.1.4
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v3.74.0
+ provider registry.terraform.io/hashicorp/cloudinit v2.2.0
+ provider registry.terraform.io/hashicorp/kubernetes v2.7.1
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/tls v3.1.0

Your version of Terraform is out of date! The latest version
is 1.1.5. You can update by downloading from https://www.terraform.io/downloads.html

• Module:
  • terraform-aws-eks 18.4.1

Reproduction

Steps to reproduce the behavior:

1. terraform init
2. terraform plan

Code Snippet to Reproduce

module "eks-ng" {
  source  = "terraform-aws-modules/eks/aws"
  version = "v18.4.1"

  cluster_name                    = local.eks_cluster_name
  cluster_version                 = "1.21"


  cluster_enabled_log_types = [
    "audit",
  ]

  cluster_addons = {
    coredns = {
      resolve_conflicts = "OVERWRITE"
    }
    kube-proxy = {}
    vpc-cni = {
      addon_version     = "v1.9.3-eksbuild.1"
      resolve_conflicts = "OVERWRITE"
    }
  }

  cluster_encryption_config = [{
    provider_key_arn = aws_kms_key.eks.arn
    resources        = ["secrets"]
  }]

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  enable_irsa = true

  # EKS Managed Node Group(s)
  eks_managed_node_group_defaults = {
    ami_type = "AL2_x86_64"
    instance_types = [
      "t3a.xlarge",
    ]
    bootstrap_extra_args = "--kubelet-extra-args '--max-pods=100'"
    iam_role_additional_policies = [
      "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
    ]

    block_device_mappings = {
      xvda = {
        device_name = "/dev/xvda"
        ebs = {
          volume_size           = 50
          volume_type           = "gp3"
          iops                  = 1000
          throughput            = 250
          encrypted             = true
          delete_on_termination = true
        }
      }
    }

    post_bootstrap_user_data = <<-EOT
      cd /tmp
      sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm
      sudo systemctl enable amazon-ssm-agent
      sudo systemctl start amazon-ssm-agent
    EOT
  }


  cluster_security_group_additional_rules = {
    egress_nodes_ephemeral_ports_tcp = {
      description      = "Cluster all egress"
      protocol         = "-1"
      from_port        = 0
      to_port          = 0
      type             = "egress"
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = ["::/0"]
    }


    ingress_self_all = {
      description = "Ingress All"
      protocol    = "-1"
      from_port   = 0
      to_port     = 0
      type        = "ingress"
      self        = true
    }
  }
  node_security_group_additional_rules = {
    egress_all = {
      description      = "Node all egress"
      protocol         = "-1"
      from_port        = 0
      to_port          = 0
      type             = "egress"
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = ["::/0"]
    }

    ingress_self_all = {
      description = "Node to node all ports/protocols"
      protocol    = "-1"
      from_port   = 0
      to_port     = 0
      type        = "ingress"
      self        = true
    }

    ingress_self_webhook_3 = {
      description                   = "Webhook #3"
      protocol                      = "TCP"
      from_port                     = 8443
      to_port                       = 8443
      type                          = "ingress"
      source_cluster_security_group = true
    }
  }
  eks_managed_node_groups = {
    worker = {
      min_size     = 1
      max_size     = 20
      desired_size = 1
    }

    infra = {
      min_size     = 0
      max_size     = 30
      desired_size = 1

      labels = {
        Tier = "infra"
      }

      taints = {
        dedicated = {
          key    = "tier"
          value  = "infra"
          effect = "NO_SCHEDULE"
        }
      }
    }
  }

  tags = local.tags
}

Expected behavior

Passed terraform plan

Actual behavior

See the terminal output

Terminal Output Screenshot(s)

Error: Cycle: module.eks-ng.module.eks_managed_node_group.module.user_data (close), module.eks-ng.module.eks_managed_node_group.output.launch_template_latest_version (expand), module.eks-ng.module.eks_managed_node_group.output.iam_role_unique_id (expand), module.eks-ng.module.eks_managed_node_group.output.node_group_resources (expand), module.eks-ng.module.eks_managed_node_group.output.iam_role_name (expand), module.eks-ng.module.eks_managed_node_group.output.security_group_arn (expand), module.eks-ng.module.eks_managed_node_group.output.launch_template_id (expand), module.eks-ng.module.eks_managed_node_group.output.node_group_status (expand), module.eks-ng.module.eks_managed_node_group.output.node_group_arn (expand), module.eks-ng.module.eks_managed_node_group.output.iam_role_arn (expand), module.eks-ng.module.eks_managed_node_group.output.launch_template_arn (expand), module.eks-ng.module.eks_managed_node_group.output.security_group_id (expand), module.eks-ng.module.eks_managed_node_group.var.taints (expand), module.eks-ng.module.eks_managed_node_group.var.subnet_ids (expand), module.eks-ng.module.eks_managed_node_group.local.launch_template_name (expand), module.eks-ng.module.eks_managed_node_group.var.use_name_prefix (expand), module.eks-ng.module.eks_managed_node_group.var.timeouts (expand), module.eks-ng.module.eks_managed_node_group.var.desired_size (expand), module.eks-ng.module.eks_managed_node_group.var.remote_access (expand), module.eks-ng.module.eks_managed_node_group.var.labels (expand), module.eks-ng.module.eks_managed_node_group.var.instance_types (expand), module.eks-ng.module.eks_managed_node_group.local.use_custom_launch_template (expand), module.eks-ng.module.eks_managed_node_group.var.ami_release_version (expand), module.eks-ng.module.eks_managed_node_group.var.ami_type (expand), module.eks-ng.module.eks_managed_node_group.var.cluster_version (expand), module.eks-ng.module.eks_managed_node_group.var.update_config (expand), module.eks-ng.module.eks_managed_node_group.var.capacity_type (expand), module.eks-ng.module.eks_managed_node_group.var.force_update_version (expand), module.eks-ng.module.eks_managed_node_group.var.max_size (expand), module.eks-ng.module.eks_managed_node_group.var.min_size (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_arn (expand), module.eks-ng.module.eks_managed_node_group.var.instance_market_options (expand), module.eks-ng.module.eks_managed_node_group.var.launch_template_name (expand), module.eks-ng.module.eks_managed_node_group.local.launch_template_name_int (expand), module.eks-ng.module.eks_managed_node_group.var.cluster_security_group_id (expand), module.eks-ng.module.eks_managed_node_group.var.security_group_description (expand), module.eks-ng.module.eks_managed_node_group.var.security_group_use_name_prefix (expand), module.eks-ng.module.eks_managed_node_group.var.vpc_id (expand), module.eks-ng.module.eks_managed_node_group.var.security_group_tags (expand), module.eks-ng.module.eks_managed_node_group.var.security_group_name (expand), module.eks-ng.module.eks_managed_node_group.local.security_group_name (expand), module.eks-ng.module.eks_managed_node_group.aws_security_group.this, module.eks-ng.module.eks_managed_node_group.var.create_security_group (expand), module.eks-ng.module.eks_managed_node_group.local.create_security_group (expand), module.eks-ng.module.eks_managed_node_group.var.security_group_rules (expand), module.eks-ng.module.eks_managed_node_group.aws_security_group_rule.this, module.eks-ng.module.eks_managed_node_group.var.enclave_options (expand), module.eks-ng.module.eks_managed_node_group.var.credit_specification (expand), module.eks-ng.module.eks_managed_node_group.var.update_launch_template_default_version (expand), module.eks-ng.module.eks_managed_node_group.var.ram_disk_id (expand), module.eks-ng.module.eks_managed_node_group.var.placement (expand), module.eks-ng.module.eks_managed_node_group.var.launch_template_use_name_prefix (expand), module.eks-ng.module.eks_managed_node_group.var.launch_template_tags (expand), module.eks-ng.module.eks_managed_node_group.var.capacity_reservation_specification (expand), module.eks-ng.module.eks_managed_node_group.var.launch_template_description (expand), module.eks-ng.module.eks_managed_node_group.var.cpu_options (expand), module.eks-ng.module.eks_managed_node_group.var.key_name (expand), module.eks-ng.module.eks_managed_node_group.var.license_specifications (expand), module.eks-ng.module.eks_managed_node_group.var.cluster_service_ipv4_cidr (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.cluster_service_ipv4_cidr (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.local.int_linux_default_user_data (expand), module.eks-ng.module.eks_managed_node_group.var.post_bootstrap_user_data (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.post_bootstrap_user_data (expand), module.eks-ng.module.eks_managed_node_group.var.pre_bootstrap_user_data (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.pre_bootstrap_user_data (expand), module.eks-ng.module.eks_managed_node_group.var.enable_bootstrap_user_data (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.enable_bootstrap_user_data (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.is_eks_managed_node_group (expand), module.eks-ng.module.eks_managed_node_group.var.platform (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.platform (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.data.cloudinit_config.linux_eks_managed_node_group, module.eks-ng.module.eks_managed_node_group.var.cluster_auth_base64 (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.cluster_auth_base64 (expand), module.eks-ng.module.eks_managed_node_group.var.cluster_endpoint (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.cluster_endpoint (expand), module.eks-ng.module.eks_managed_node_group.var.cluster_name (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.cluster_name (expand), module.eks-ng.module.eks_managed_node_group.var.bootstrap_extra_args (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.bootstrap_extra_args (expand), module.eks-ng.module.eks_managed_node_group.var.user_data_template_path (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.user_data_template_path (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.var.create (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.local.platform (expand), module.eks-ng.module.eks_managed_node_group.module.user_data (expand), module.eks-ng.module.eks_managed_node_group.module.user_data.output.user_data (expand), module.eks-ng.module.eks_managed_node_group.var.ebs_optimized (expand), module.eks-ng.module.eks_managed_node_group.var.kernel_id (expand), module.eks-ng.module.eks_managed_node_group.var.launch_template_default_version (expand), module.eks-ng.module.eks_managed_node_group.var.vpc_security_group_ids (expand), module.eks-ng.module.eks_managed_node_group.var.network_interfaces (expand), module.eks-ng.module.eks_managed_node_group.var.elastic_inference_accelerator (expand), module.eks-ng.module.eks_managed_node_group.var.block_device_mappings (expand), module.eks-ng.module.eks_managed_node_group.var.ami_id (expand), module.eks-ng.module.eks_managed_node_group.var.enable_monitoring (expand), module.eks-ng.module.eks_managed_node_group.var.elastic_gpu_specifications (expand), module.eks-ng.module.eks_managed_node_group.var.create_launch_template (expand), module.eks-ng.module.eks_managed_node_group.var.metadata_options (expand), module.eks-ng.module.eks_managed_node_group.var.disable_api_termination (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_additional_policies (expand), module.eks-ng.module.eks_managed_node_group.data.aws_caller_identity.current, module.eks-ng.module.eks_managed_node_group.var.cluster_ip_family (expand), module.eks-ng.module.eks_managed_node_group.local.cni_policy (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_attach_cni_policy (expand), module.eks-ng.module.eks_managed_node_group.local.iam_role_policy_prefix (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_use_name_prefix (expand), module.eks-ng.module.eks_managed_node_group.var.tags (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_permissions_boundary (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_description (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_tags (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_path (expand), module.eks-ng.module.eks_managed_node_group.data.aws_partition.current, module.eks-ng.module.eks_managed_node_group.var.create_iam_role (expand), module.eks-ng.module.eks_managed_node_group.data.aws_iam_policy_document.assume_role_policy, module.eks-ng.module.eks_managed_node_group.var.name (expand), module.eks-ng.module.eks_managed_node_group.var.iam_role_name (expand), module.eks-ng.module.eks_managed_node_group.local.iam_role_name (expand), module.eks-ng.module.eks_managed_node_group.aws_iam_role.this, module.eks-ng.module.eks_managed_node_group.var.create (expand), module.eks-ng.module.eks_managed_node_group.aws_iam_role_policy_attachment.this, module.eks-ng.module.eks_managed_node_group.aws_launch_template.this, module.eks-ng.module.eks_managed_node_group.var.launch_template_version (expand), module.eks-ng.module.eks_managed_node_group.local.launch_template_version (expand), module.eks-ng.module.eks_managed_node_group.aws_eks_node_group.this, module.eks-ng.module.eks_managed_node_group.output.node_group_id (expand), module.eks-ng.aws_eks_addon.this, module.eks-ng.output.cluster_addons (expand), module.eks-ng.var.eks_managed_node_groups (expand), module.eks-ng.module.eks_managed_node_group (expand), module.eks-ng.module.eks_managed_node_group.var.disk_size (expand), module.eks-ng.module.eks_managed_node_group (close)

Additional context

[bryantbiggs]

After further testing, I am not able to reproduce this issue

[bryantbiggs]

closing due to lack of reproduction

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.