Community Note

Voting for Prioritization

• Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

• Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
• For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
• Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

In the case of pilot light multi-region deployments, I think the approach presented here would generally work.

There are some interesting concerns around resource types that inherently support multi-region via primary/replica attributes or global attributes.

I also would want to address how Terraform would destroy resources in a region if the practitioner removes a region. Say they want to leave ap-south-1, how could they ensure that all resources were removed?

See #27758 for some good comments.

@AdamTylerLynch In the case of resources that support multiple regions, I think that this will simplify those implementations. Just looking at the example of aws_dynamodb_table_replica. Currently the code is creating a new session to the target region, where with this update it could instead select the correct client for the target region and remove the need for configuring these clients within each multi-region resource.

If a user wanted to remove all resources from a region, they would need to remove the terraform configuration for those resources and allow terraform to destroy them, or run a targeted destroy.

If a user wanted to move from say ap-south-1 to us-west-1, they would simply update the region value for each resource (hopefully they would have the target region as a local so it only requires updating a single value), then terraform would handle destroying and recreating them all in the new region. Since the value is determined at the region, dynamic values can be used like local, for_each and count.

In the below basic example, a user would simply remove the region from the local.regions list, and terraform would handle destroying the resources.

locals {
  regions = [
  "us-east-1",
  "us-east-2"]
}


resource "aws_lb" "test" {
  for_each           = teset(local.regions)
  region              = each.key
  name               = "test-lb-tf"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.allow_tls.id]
  subnets            = data.aws_subnets.public.ids

  tags = {
    Environment = "production"
  }
}

In the event that the user updated the region value for the provider, terraform would behave as it does today and simply error out stating that it could not find the resources. A destroy would need to be performed before cleaning updating the default region for the provider.

One thing I am still trying to figure out how to do is in the event a user was providing a region value and decides to remove that value, The provider needs to be smart enough to know to default to the provider default region. Still working on how to technically make that work.

@AdamTylerLynch I have additionally added a central flex function in order to return a proper error to the client in the event they are attempting to deploy to a region that is not in the allowed_regions provider settings. This should help force the behavior of either updating the region or destroying the resource before the region is removed from the allowed list.

Thanks for all the thought and work you've put into this, @brittandeyoung. We're going to have some discussions on our team around the approach to take with this.

We see some definite benefits to multiple regions with one providers, but also some areas for caution, especially around how regionally-separated AWS services are.

Thanks for all the thought and work you've put into this, @brittandeyoung. We're going to have some discussions on our team around the approach to take with this.

We see some definite benefits to multiple regions with one providers, but also some areas for caution, especially around how regionally-separated AWS services are.

@gdavison great! I will wait to hear from the provider team before I put any more work into this.

I personally like this idea.

1. It could help with performance since you wouldn't need multiple providers (especially for people who support a lot of regions)
2. It would simplify configurations 🎉 (e.g., replications, clusters, modules)
3. There may be other things that could work this way (I dunno but maybe assume_role, profile?)

Hello! I work on Terraform Core rather than on the AWS provider so I'm just an interested onlooker here and not intending to speak on behalf of the AWS provider team. I've had a quiet hope for the AWS provider to adopt a model like this for a long time, so I'm very excited to see this proposal/prototype. 😀

This new approach could potentially also help with another long-standing oddity: if one changes the region argument in the provider configuration today after some remote objects already exists then the provider typically tries to refresh the existing objects in the wrong region, gets a "not found" error, and assumes that the objects have been deleted. Unless the user checks the provider's work and rejects the plan, they could potentially end up creating duplicate objects in the new region and leaving behind forgotten objects in the old region.

Here's one way I could imagine improving that region change situation:

• The region argument in each resource type is declared as Optional: true, Computed: true, so the provider can populate it if the module author doesn't.
• If per-resource region is unset in the configuration, the provider automatically populates it during planning (i.e. CustomizeDiff in the SDK) with the same region as configured in the corresponding provider "aws" block, which now serves as the "default" region for any resource that doesn't specify one.
  • To allow upgrading from state objects that were created with earlier versions of the provider, each resource type would need an upgrade rule to retrofit a value for region into any resource instance that doesn't already have one, based on the current provider configuration. For correct results, the author would need to leave the provider-level region unchanged while resolving the upgrade so that the correct per-resource value can be retrofitted and then the configuration and state will match during the first plan after upgrading.
  • I think this rule would effectively replace the empty-region-handling DiffSuppressFunc in the current PR, since there would no longer be any case where a resource would have no region populated except during the state upgrade step, which always runs before any planning steps.
• When refreshing existing objects and when destroying, the provider uses the region attribute on the resource to decide which region to make the query to, ignoring the "default region" in the provider "aws" block.
• When diffing, if a resource instance region in configuration is different than the prior state, signal that replacement is required ("Force new" in SDK terminology) so that Terraform will propose to delete the object in the old region and create a new object in the new region.
• If the region in the provider "aws" block changes when remote objects already exist, the provider treats that as a change to desired state of all resources which don't explicitly set region, making it now possible to mass-migrate many objects from one region to another by replacement.

The above is just one design idea and I listed out these details only in case they are useful; some parts of this are probably challenging to implement with the current capabilities of the plugin SDK, but I think it is all valid within and consistent with the spirit of Terraform's resource instance change lifecycle. The main thing I'd love to see, details aside, is that changing region in the provider "aws" block would no longer cause the provider to be confused by "not found" errors and instead have it treat the region change like a normal change to desired state.

Again, I'm not speaking on behalf of the AWS provider team and so please don't take any actions based on this until the provider team has said something about it! I don't want anyone to spend time on implementing something like this if the provider team would eventually oppose it on principle anyway.

Anything I could do to help move this along