Possible to reference ASGs created by EKS managed node groups?

[nick-place]

Is your request related to a new offering from AWS?

No.

Is your request related to a problem? Please describe.

We are using eks managed node groups and I am trying to see if there is a way we can reference the ASGs the module creates for each node group. We use the ARNs from the ASGs for our cluster autoscaler IAM policy and the names from the ASGs to tag our ASGs so cluster autoscaler can scale to zero.

It appears the eks-managed-node-group submodule creates the aws_launch_template resource but I do not see an aws_autoscaling_group resource created within that submodule. I do see one created when using the self-managed-node-group submodule, though, and the respective outputs available.

Is it possible to reference the ASGs created when using eks managed node groups from the module? I don't see the ASGs in our terraform state (just the launch templates) so I'm not sure this is possible, but wanted to verify. I searched through the docs and issues here before posting so apologies if this is an obvious solution I am overlooking or if I am misunderstanding how these ASGs are created using the module and submodules.

Describe the solution you'd like.

Ideally I would like to reference the ASGs as outputs from the module rather than pulling their info in from a data source.

Describe alternatives you've considered.

We are currently looking these ASGs up using a data source:

data "aws_autoscaling_groups" "eks_autoscaling_groups" {
  filter {
    name   = "tag:eks:cluster-name"
    values = ["${data.aws_iam_account_alias.current.account_alias}-gitlab-ci"]
  }
}

To tag our autoscaling groups:

resource "aws_autoscaling_group_tag" "scale_to_zero_tag" {

  for_each = toset(data.aws_autoscaling_groups.eks_autoscaling_groups.names)

  autoscaling_group_name = each.value

  tag {
    key   = "k8s.io/cluster-autoscaler/node-template/label/eks.amazonaws.com/capacityType"
    value = "ON_DEMAND"

    propagate_at_launch = true
  }
}

This works, but for example, if we change something that creates a new node group, we won't pick up that new ASG until our pipeline runs a second time.

We also use the ASG ARNs from the data source for our cluster autoscaler policy:

resource "aws_iam_role_policy" "cluster_autoscaler_policy" {
  name = "eks-cluster-autoscaler-policy"
  role = aws_iam_role.cluster_autoscaler_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = [
          "autoscaling:DescribeAutoScalingGroups",
          "autoscaling:DescribeAutoScalingInstances",
          "autoscaling:DescribeLaunchConfigurations",
          "autoscaling:DescribeTags",
          "ec2:DescribeInstanceTypes"
        ]
        Effect   = "Allow"
        Resource = "*"
      },
      {
        Action = [
          "autoscaling:SetDesiredCapacity",
          "autoscaling:TerminateInstanceInAutoScalingGroup",
        ]
        Effect   = "Allow"
        Resource = "${data.aws_autoscaling_groups.eks_autoscaling_groups.arns}"
      },
    ]
  })
}

But again, if a change is made that requires new ASGs, these won't be picked up until our pipeline is run again.

Additional context

source  = "terraform-aws-modules/eks/aws"
version = "18.9.0"

$ terraform version
Terraform v1.1.5
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v4.1.0
+ provider registry.terraform.io/hashicorp/cloudinit v2.2.0
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/tls v3.1.0

[bryantbiggs]

yes, this is already exposed since the whole map of attributes from node groups is provided as an output

[nick-place]

Thanks for the quick reply.

output "eks_managed_node_groups" {
  description = "Map of attribute maps for all EKS managed node groups created"
  value       = module.eks_managed_node_group
}

I'll dig into using that map, thanks.

[bryantbiggs]

it will be something like

locals {
	asg_names = [
		for group in module.eks.eks_managed_node_groups: [
			for asg in group.node_group_resources.autoscaling_groups: 
				asg.name
	]
}

I did not test this but just a rough guess looking at the docs

[antonbabenko]

This issue has been resolved in version 18.12.0 🎉

[rajarajeshwaran-vijayakumar]

Hi, I need to reference ASG's arn to create a event pattern to create cloudwatch event bridge. how do i retrieve the ARN of ASG created for EKS managed node groups besides the name ?

[bryantbiggs]

you can build the ARN yourself using the names provided by the module output

[rajarajeshwaran-vijayakumar]

ASG's ARN looks something like this .. arn:aws:autoscaling:us-west-2:{AWS-Account}:autoScalingGroup:{AWS HASHKEY}:autoScalingGroupName/{ASG-name}

How do i retrieve the aws hash key in order to form the complete ARN. ?

[bryantbiggs]

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/autoscaling_group

[rajarajeshwaran-vijayakumar]

@bryantbiggs while using the data source mentioned above, i was able to retrieve the ASG's ARN of EKS managed node groups. however, whenever i provision a new cluster the data source not retrieving the data from ASG and fails at the first time, and works if i re run terraform plan and apply again next time. any idea why this is happening ?

[bryantbiggs]

Because the ASG is created after applying - the EKS managed node group service creates the ASG so you can't query something until it exists

[rajarajeshwaran-vijayakumar]

I have added 'depends on' on data source block to make sure it queries after the EKS managed node groups and cluster is cleated and it did not help.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.