Desired_capiacity is not updatable after init deployment

[Staggerlee011]

After deploying an EKS Cluster i updated the min and desired values. Terraform will fail saying that min_capacity cannot be more than desired. If i update just the desired_capacity value terraform shows no changes are needed.

• bug report
• feature request
• [* ] support request - read the FAQ first!
• kudos, thank you, warm fuzzy

What is the current behavior?

desired_capacity value is ignored after initial deployment and cannot be changed via terraform.

What's the expected behavior?

desired_capacity value is used

Any other relevant info

Terraform: 1.0.3
EKS Module: 17.1.0

node_groups = {
    default = {

      name = var.default_nodegroup

      desired_capacity = 3
      max_capacity     = 10
      min_capacity     = 1

      instance_types = ["t3.medium"]
      capacity_type  = "ON_DEMAND"


      launch_template_id      = aws_launch_template.managed_ng_lt.id
      launch_template_name    = aws_launch_template.managed_ng_lt.name
      launch_template_version = aws_launch_template.managed_ng_lt.latest_version


      additional_tags = {
        nodegroup = var.default_nodegroup
      }

      k8s_labels = {
        nodegroup = var.default_nodegroup
      }
    }
  }

Changing desired_capacity from 1 to 2 returns:

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Updating min_capacity from 1 to 2 and desired_capacity from 1 to 2 errors:

│ Error: error updating EKS Node Group (eks_data-science:default_mng_nodegroup) config: InvalidParameterException: Minimum capacity 2 can't be greater than desired size 1
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "44c7e001-f5ce-4bd7-b7e9-55500011841f"
│   },
│   ClusterName: "eks_data-science",
│   Message_: "Minimum capacity 2 can't be greater than desired size 1",
│   NodegroupName: "default_mng_nodegroup"
│ }
│ 
│   with module.eks.module.node_groups.aws_eks_node_group.workers["default"],
│   on .terraform/modules/eks/modules/node_groups/node_groups.tf line 1, in resource "aws_eks_node_group" "workers":
│    1: resource "aws_eks_node_group" "workers" {

Terraform plan:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.eks.module.node_groups.aws_eks_node_group.workers["default"] will be updated in-place
  ~ resource "aws_eks_node_group" "workers" {
        id                   = "eks_cluster:default_mng_nodegroup"
        # (16 unchanged attributes hidden)


      ~ scaling_config {
          ~ min_size     = 1 -> 2
            # (2 unchanged attributes hidden)
        }

        # (2 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

[daroga0002]

meet exactly same problem 5 minutes ago, debugging it now but in sure it is not related to module itself.

Looks like a bug in terraform AWS provider, I will post a update when I will find a root cause

[daroga0002]

Ok, I know this is not updating desired instances number. This is because:

terraform-aws-eks/modules/node_groups/node_groups.tf

Line 102 in a9b8455

ignore_changes = [scaling_config.0.desired_size]

where we ignore desired size. Purpose of this is that we allow Cluster autoscaler to manage node groups. So if you want to influence a desired count you must modify this from WEB UI/AWS cli. To make this from terraform you must modify minimum size.

I think we must update docs for this input

[miketitus]

Minimum size cannot be increased using Terraform 0.12 (I know, I know) with AWS provider v3.59.0:

      ~ scaling_config {
            desired_size = 1
          ~ max_size     = 1 -> 2
          ~ min_size     = 1 -> 2
        }
...
Error: error updating EKS Node Group (******:******) config: InvalidParameterException: Minimum capacity 2 can't be greater than desired size 1
{
  RespMetadata: {
    StatusCode: 400,
    RequestID: "c2ef0a23-2408-4730-a948-379e39404677"
  },
  ClusterName: "******",
  Message_: "Minimum capacity 2 can't be greater than desired size 1",
  NodegroupName: "******"
}

[malcolm061990]

Ok, I know this is not updating desired instances number. This is because:

terraform-aws-eks/modules/node_groups/node_groups.tf

Line 102 in a9b8455

ignore_changes = [scaling_config.0.desired_size]

where we ignore desired size. Purpose of this is that we allow Cluster autoscaler to manage node groups. So if you want to influence a desired count you must modify this from WEB UI/AWS cli. To make this from terraform you must modify minimum size.

I think we must update docs for this input

This will update only minimum size. But the issue with changing the desired size still exists.

[tobamaestro]

Ok, I know this is not updating desired instances number. This is because:

terraform-aws-eks/modules/node_groups/node_groups.tf

Line 102 in a9b8455

ignore_changes = [scaling_config.0.desired_size]

where we ignore desired size. Purpose of this is that we allow Cluster autoscaler to manage node groups. So if you want to influence a desired count you must modify this from WEB UI/AWS cli. To make this from terraform you must modify minimum size.
I think we must update docs for this input

This will update only minimum size. But the issue with changing the desired size still exists.

Indeed. Changing min/max capacity also won't be possible if the previously set desired_capacity is for example lower than the new min_capacity.

This is still an ugly issue and for now I can only fix it manually in AWS console and align the values in Terraform.

[mrmarcsmith]

We ran into this too, totally unexpected behavior. This Issue looks like a good solution for at least half the functionality. I would love to be able to intentionally >>increase<< my desired size through terraform as well though. is it possible to only ignore the change if the desired_size is lower than current size? that would prevent deleting nodes while allowing the intentional increase of size for time you know an influx of traffic is coming.

[bryantbiggs]

unfortunately, no - once an attribute is ignored, Terraform completely disregards any changes to it.

I haven't tried this myself, but for the situation you are describing - I would suggest just increasing the min_size

[nickjj]

I haven't tried this myself, but for the situation you are describing - I would suggest just increasing the min_size

This doesn't work. We're still stuck not being able to use Terraform to adjust the cluster size.

Here's an example scenario:

You spin up an initial cluster with:

• Desired: 2
• Min: 2
• Max: 3

Now let's say you want to bump up the min size to 3 to add capacity to your cluster so you adjust your Terraform config to set the min to 3.

When you apply it, you'll get an error saying the desired size can't be less than the minimum. Even if you change both the min and desired to 3 it doesn't matter because this EKS module ignores the desired change meaning from Terraform's POV the desired state is never going to be different than what it was when you first created your cluster (2 in this case which is < 3).

In the end this leaves us with having to use the AWS web UI and not Terraform to adjust the size of the cluster. That or we don't use this EKS module for creating the worker nodes and instead use the raw https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group resource where we can set the scaling_config manually and not ignore the desired size but instead keep it in sync with the min. This is how the AWS web UI works, if you bump the min it'll auto-bump the desired size (even client side in the input box).

[clayrisser]

I'm having the same issue.

[Speculor]

Still an issue and extremely problematic for anyone using EKS module

[bryantbiggs]

Still an issue and extremely problematic for anyone using EKS module

It is not extremely problematic, it is a design decision the module has taken. The majority of Kubernetes/EKS users utilize some form of autoscaling and without variable support for ignore_changes by Terraform core, thats what we currently have

[bryantbiggs]

https://github.com/bryantbiggs/eks-desired-size-hack