Does 'desired_capacity_type' work at all? #31521

UryDanko · 2023-05-22T12:06:25Z

Description

terraform: 1.4.6
terraform-provider-aws: 4.67.0
Following configuration:

resource "aws_autoscaling_group" "asg" {
  name                      = var.name
  max_size                  = 5
  min_size                  = 1
  desired_capacity          = 2
  desired_capacity_type     = "memory-mib"
...
}

has green terraform plan:

+ resource "aws_autoscaling_group" "asg" {
      + arn                       = (known after apply)
      + availability_zones        = (known after apply)
      + capacity_rebalance        = true
      + default_cooldown          = (known after apply)
      + desired_capacity          = 2
      + desired_capacity_type     = "memory-mib"

however on apply it fails to create with the following error:

│ Error: setting Auto Scaling Group (jenkins-asg) initial capacity: ValidationError: The DesiredCapacityType field is only allowed in mixed instances groups containing InstanceRequirements overrides. Please include a MixedInstancesPolicy with InstanceRequirements for your group and try again.
│ 	status code: 400, request id: 8e6047f6-9b1c-49c1-b227-0dd8db29e460
│
│   with module.jenkins-asg[0].aws_autoscaling_group.asg,
│   on modules/flexible-asg/main.tf line 77, in resource "aws_autoscaling_group" "asg":
│   77: resource "aws_autoscaling_group" "asg" {

However if I follow the error and move desired_capacity_type to another place:

resource "aws_autoscaling_group" "asg" {
  name                      = var.name
  max_size                  = 5
  min_size                  = 1
  desired_capacity          = 2
 mixed_instances_policy {
  launch_template {
      override {
        instance_requirements {
              desired_capacity_type     = "memory-mib"
}
}
}
...
}

it fails on plan with

Error: Unsupported argument
│
│   on modules/flexible-asg/main.tf line 106, in resource "aws_autoscaling_group" "simple":
│  106:           desired_capacity_type  =  "memory-mib"
│
│ An argument named "desired_capacity_type" is not expected here.

References

doc:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group#desired_capacity_type

implemented in:
#26641

Would you like to implement a fix?

None

The text was updated successfully, but these errors were encountered:

github-actions · 2023-05-22T12:06:36Z

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

justinretzolk · 2023-05-22T18:13:02Z

Hey @UryDanko 👋 Thank you for taking the time to raise this! My read on the error is a bit different than yours.

Based on the resource's schema, your initial configuration had the desired_capacity_type at the correct level, so the error message you received when you moved it to mixed_instances_policy.launch_template.override is what I would expect to see. You'll need to move that argument back up to the top level of the resource.

What the error message you initially received appears to be indicating is that the desired_capacity_type argument can only be set if you also include a value for the argument mixed_instances_policy.launch_template.override.instance_requirements.

UryDanko · 2023-05-23T04:50:21Z

Hi @justinretzolk , thank you for your reply. Indeed, on the upper level, this property looks better. However, I didn't provide the full configuration in the previous message. I had the block you mentioned. Original configuration looks like:

resource "aws_autoscaling_group" "asg" {
  name                      = var.name
  max_size                  = 5
  min_size                  = 1
  desired_capacity          = 2
  desired_capacity_type     = "memory-mib"
...
  mixed_instances_policy {
    launch_template {
      launch_template_specification {
        launch_template_id = aws_launch_template.asg.id
        version            = "$Latest"
      }
      override {
        instance_requirements {
          memory_mib {
            min = var.memory_mib_min
          }
          vcpu_count {
            min = var.vcpu_count_min
          }
        }
      }
    }

    instances_distribution {
      on_demand_base_capacity                  = var.on_demand_base_capacity
      on_demand_allocation_strategy            = var.on_demand_allocation_strategy
      spot_allocation_strategy                 = var.spot_allocation_strategy
      on_demand_percentage_above_base_capacity = var.on_demand_percentage_above_base_capacity
    }
  }

some parameters I omitted for brevity.

justinretzolk · 2023-05-24T21:46:43Z

Hey @UryDanko 👋 Thanks for that additional information! That certainly looks a bit more bug-y now that you've called that out. Are you able to supply debug logs (redacted as necessary) as well, so that the team has the necessary information in order to look into this?

UryDanko · 2023-05-31T08:57:26Z

Hi @justinretzolk ,
apologies for the delay. I guess these logs you would like to see?

2023-05-30T10:27:18.7699824Z 2023-05-30T10:27:18.769Z [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for module.jenkins-deploy-bot-new-asg[0].aws_launch_template.asg, but we are tolerating it because it is using the legacy plugin SDK.
2023-05-30T10:27:18.7712525Z �[0m�[1mmodule.jenkins-deploy-bot-new-asg[0].aws_launch_template.asg: Creating...�[0m�[0m
2023-05-30T10:27:18.7714028Z 2023-05-30T10:27:18.769Z [INFO]  Starting apply for module.jenkins-deploy-bot-new-asg[0].aws_launch_template.asg
2023-05-30T10:27:18.7715666Z 2023-05-30T10:27:18.771Z [DEBUG] module.jenkins-deploy-bot-new-asg[0].aws_launch_template.asg: applying the planned Create change
2023-05-30T10:27:18.9638960Z 2023-05-30T10:27:18.952Z [WARN]  Provider "provider[\"registry.terraform.io/hashicorp/aws\"]" produced an unexpected new value for module.jenkins-deploy-bot-new-asg[0].aws_launch_template.asg, but we are tolerating it because it is using the legacy plugin SDK.
2023-05-30T10:27:18.9648520Z �[0m�[1mmodule.jenkins-deploy-bot-new-asg[0].aws_launch_template.asg: Creation complete after 0s [id=lt-0adec0c35c74251d7]�[0m
2023-05-30T10:27:18.9784396Z 2023-05-30T10:27:18.977Z [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for module.jenkins-deploy-bot-new-asg[0].aws_autoscaling_group.asg, but we are tolerating it because it is using the legacy plugin SDK.
2023-05-30T10:27:18.9795410Z �[0m�[1mmodule.jenkins-deploy-bot-new-asg[0].aws_autoscaling_group.asg: Creating...�[0m�[0m
2023-05-30T10:27:18.9796335Z 2023-05-30T10:27:18.979Z [INFO]  Starting apply for module.jenkins-deploy-bot-new-asg[0].aws_autoscaling_group.asg
2023-05-30T10:27:18.9815560Z 2023-05-30T10:27:18.981Z [DEBUG] module.jenkins-deploy-bot-new-asg[0].aws_autoscaling_group.asg: applying the planned Create change
2023-05-30T10:27:20.9730160Z 2023-05-30T10:27:20.972Z [ERROR] vertex "module.jenkins-deploy-bot-new-asg[0].aws_autoscaling_group.asg" error: setting Auto Scaling Group (jenkins-deploy-bot-integration-***-asg) initial capacity: ValidationError: The DesiredCapacityType field is only allowed in mixed instances groups containing InstanceRequirements overrides. Please include a MixedInstancesPolicy with InstanceRequirements for your group and try again.
2023-05-30T10:27:23.3892247Z       "module": "module.jenkins-deploy-bot-new-asg[0]",
2023-05-30T10:27:23.3931251Z       "module": "module.jenkins-deploy-bot-new-asg[0]",
2023-05-30T10:27:23.4016379Z             "module.jenkins-deploy-bot-new-asg.aws_launch_template.asg",
2023-05-30T10:27:23.4016913Z             "module.jenkins-deploy-bot-new-asg.data.aws_ami.asg"
2023-05-30T10:27:23.4018798Z       "module": "module.jenkins-deploy-bot-new-asg[0]",
2023-05-30T10:27:23.4088470Z             "module.jenkins-deploy-bot-new-asg.data.aws_ami.asg"
2023-05-30T10:27:24.3316582Z �[31m│�[0m �[0m�[0m  with module.jenkins-deploy-bot-new-asg[0].aws_autoscaling_group.asg,

this is just the last part of the logs, immediately before the error occurred. If you need to see the full log (related to this ASG), please let me know.

justinretzolk · 2023-06-02T20:35:48Z

Hey @UryDanko, thanks for the update! Unfortunately, I don't believe that snippet will quite have the information we'll need, so if you can provide the whole thing, that would be helpful.

UryDanko · 2023-06-06T14:11:39Z

Hi @justinretzolk , I attached the compressed debug log. Did some cleaning though, removed not relevant entries about the other components. Please let me know if you have any questions.

asg_tf_debug_sanitized.log.zip

UryDanko · 2023-06-14T10:53:26Z

terraform-provider-aws: 5.3.0

Error: -14T10:45:28.572Z [ERROR] provider.terraform-provider-aws_v5.3.0_x5: Response contains error diagnostic: @caller=github.com/hashicorp/terraform-plugin-go@v0.15.0/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_severity=ERROR @module=sdk.proto diagnostic_detail= diagnostic_summary="setting Auto Scaling Group (xxx-asg) initial capacity: ValidationError: The DesiredCapacityType field is only allowed in mixed instances groups containing InstanceRequirements overrides. Please include a MixedInstancesPolicy with InstanceRequirements for your group and try again.
	status code: 400, request id: 512226ec-3de7-40cc-9ded-06192a9bba5c" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=6385d2a7-0001-1552-14c1-2ccada4467ce tf_resource_type=aws_autoscaling_group tf_rpc=ApplyResourceChange timestamp=2023-06-14T10:45:28.572Z
Error: -14T10:45:28.589Z [ERROR] vertex "module.xxx-asg[0].aws_autoscaling_group.asg" error: setting Auto Scaling Group (xxx-asg) initial capacity: ValidationError: The DesiredCapacityType field is only allowed in mixed instances groups containing InstanceRequirements overrides. Please include a MixedInstancesPolicy with InstanceRequirements for your group and try again.
	status code: 400, request id: 512226ec-3de7-40cc-9ded-06192a9bba5c
...
│ Error: setting Auto Scaling Group (xxx-asg) initial capacity: ValidationError: The DesiredCapacityType field is only allowed in mixed instances groups containing InstanceRequirements overrides. Please include a MixedInstancesPolicy with InstanceRequirements for your group and try again.
│ 	status code: 400, request id: 512226ec-3de7-40cc-9ded-06192a9bba5c
│ 
│   with module.xxx-asg[0].aws_autoscaling_group.asg,
│   on modules/flexible-asg/main.tf line 77, in resource "aws_autoscaling_group" "asg":
│   77: resource "aws_autoscaling_group" "asg" {

so the same error

snolan-amount · 2023-07-17T18:10:56Z

I have seen this also cause an error where I cannot apply because the plan wants to remove the desired_capacity_type and the AWS API throws this error

Error: updating Auto Scaling Group (my_autoscaling_group): InvalidParameter: 1 validation error(s) found. - minimum field size of 1, UpdateAutoScalingGroupInput.DesiredCapacityType.

In my case I had never enabled or used attribute-base instances - we made a minor update to an instance type in the AWS console and that caused the API to now return a value for DesiredCapacityType, which in turn is causing the provider to try to remove it. The only solution has been to add an ignore for desired_capacity_type in the terraform, otherwise we get the error if we do not set desired_capacity_type = "units" or we get a phantom change to desired_capacity_type every time we plan because the provider wants to set it but it isn't in use.

UryDanko · 2023-07-26T13:38:01Z

I just checked and for
terraform-aws-provider: 5.9.0
terraform: 1.5.3 the issue still exists.

@justinretzolk , did you have a chance to have a look into logs I attached?

shantanugadgil · 2023-08-09T13:54:35Z

I have seen this also cause an error where I cannot apply because the plan wants to remove the desired_capacity_type and the AWS API throws this error
Error: updating Auto Scaling Group (my_autoscaling_group): InvalidParameter: 1 validation error(s) found. - minimum field size of 1, UpdateAutoScalingGroupInput.DesiredCapacityType.
In my case I had never enabled or used attribute-base instances - we made a minor update to an instance type in the AWS console and that caused the API to now return a value for DesiredCapacityType, which in turn is causing the provider to try to remove it. The only solution has been to add an ignore for desired_capacity_type in the terraform, otherwise we get the error if we do not set desired_capacity_type = "units" or we get a phantom change to desired_capacity_type every time we plan because the provider wants to set it but it isn't in use.

Same issue observed with:

Terraform v1.5.5
AWS provider v5.11.0

if not specified, the default value for desired_capacity_type should be string units .

Currently I am observing the following diff:

 # aws_autoscaling_group.asg will be updated in-place
  ~ resource "aws_autoscaling_group" "asg" {   
      - desired_capacity_type     = "units" -> null
...

UryDanko · 2023-10-26T12:30:39Z

terraform-aws-provider: 5.22.0
terraform: 1.6.2

after some experiments I found out that everything works fine. I can use desired_capacity, desired_capacity_type, allowed_instance_types - and everything seems to be fine. Just till the moment I add initial_lifecycle_hook block into aws_autoscaling_group. After that I get an error:

 Error: setting Auto Scaling Group (jenkins-***-asg) initial capacity:
 ValidationError: The DesiredCapacityType field is only allowed in mixed instances groups
  containing InstanceRequirements overrides. Please include a MixedInstancesPolicy with InstanceRequirements

as soon as I remove this block - the pipeline is green again. I tried to create/update ASG - the result is the same.

if I add instead aws_autoscaling_lifecycle_hook block - it works good as well.

sff0219 · 2024-02-01T16:16:34Z

We have noticed a similar behavior as @snolan-amount described.

We set up an aws_autoscaling_group few weeks ago. During our latest development, we didn't change anything in the terraform template of this ASG. However, while applying the changes with terraform apply, we receive the following error.

# module.ecs_cluster.aws_autoscaling_group.xxx_asg will be updated in-place

 ~ resource "aws_autoscaling_group" "xxx_asg" {
 - desired_capacity_type            = "units" -> null
 id                               = "xxx-asg"
 name                             = "xxx-asg"
 # (26 unchanged attributes hidden)
  
 # (2 unchanged blocks hidden)
}

Error: updating Auto Scaling Group (xxx-asg): InvalidParameter: 1 validation error(s) found.
- minimum field size of 1, UpdateAutoScalingGroupInput.DesiredCapacityType.

It looks like terraform automatically reset desired_capacity_type to null, if it is not explicitly defined.

terraform-aws-provider: 5.24.0
terraform: 1.6.3

piotr-jagiello · 2024-09-05T17:24:04Z

I'm having this issue or something very similar - I can't create an autoscaling group when setting both desired_capacity_type and instance_requirements (I'm getting that nonsense error "Please include a MixedInstancesPolicy with InstanceRequirements for your group and try again" which is literally what I just did).

A workaround that somehow seems to work is removing both desired_capacity_type and instance_requirements during the first apply, once that goes through you can bring those values back and apply again and everything works fine from there. Very annoying.

UryDanko added the needs-triage Waiting for first response or review from a maintainer. label May 22, 2023

github-actions bot added the service/autoscaling Issues and PRs that pertain to the autoscaling service. label May 22, 2023

justinretzolk added waiting-response Maintainers are waiting on response from community or contributor. and removed needs-triage Waiting for first response or review from a maintainer. labels May 22, 2023

github-actions bot removed the waiting-response Maintainers are waiting on response from community or contributor. label May 23, 2023

justinretzolk added bug Addresses a defect in current functionality. waiting-response Maintainers are waiting on response from community or contributor. labels May 24, 2023

github-actions bot removed the waiting-response Maintainers are waiting on response from community or contributor. label May 31, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Does 'desired_capacity_type' work at all? #31521

Does 'desired_capacity_type' work at all? #31521

UryDanko commented May 22, 2023

github-actions bot commented May 22, 2023

justinretzolk commented May 22, 2023

UryDanko commented May 23, 2023

justinretzolk commented May 24, 2023

UryDanko commented May 31, 2023

justinretzolk commented Jun 2, 2023

UryDanko commented Jun 6, 2023

UryDanko commented Jun 14, 2023

snolan-amount commented Jul 17, 2023

UryDanko commented Jul 26, 2023

shantanugadgil commented Aug 9, 2023

UryDanko commented Oct 26, 2023 •

edited

Loading

sff0219 commented Feb 1, 2024 •

edited

Loading

piotr-jagiello commented Sep 5, 2024

Does 'desired_capacity_type' work at all? #31521

Does 'desired_capacity_type' work at all? #31521

Comments

UryDanko commented May 22, 2023

Description

References

Would you like to implement a fix?

github-actions bot commented May 22, 2023

Community Note

justinretzolk commented May 22, 2023

UryDanko commented May 23, 2023

justinretzolk commented May 24, 2023

UryDanko commented May 31, 2023

justinretzolk commented Jun 2, 2023

UryDanko commented Jun 6, 2023

UryDanko commented Jun 14, 2023

snolan-amount commented Jul 17, 2023

UryDanko commented Jul 26, 2023

shantanugadgil commented Aug 9, 2023

UryDanko commented Oct 26, 2023 • edited Loading

sff0219 commented Feb 1, 2024 • edited Loading

piotr-jagiello commented Sep 5, 2024

UryDanko commented Oct 26, 2023 •

edited

Loading

sff0219 commented Feb 1, 2024 •

edited

Loading