Terragrunt is a thin wrapper for the Terraform client that provides simple locking mechanisms so that multiple people can collaborate on the same Terraform state without overwriting each other's changes. You can choose from one of the supported locking mechanisms:
- Git: Use your version control system to acquire and release locks. This is a totally free solution that should work for most teams.
- DynamoDB: Use Amazon's DynamoDB to acquire and release locks. This is a great option for teams already using AWS. DynamoDB is part of the AWS free tier, so this option should also be free.
When you use Terraform to provision infrastructure, it records the state of your infrastructure in state
files. In order to make changes to your infrastructure, everyone on your
team needs access to these state files. You could check the files into version control (not a great idea, as the state
files may contain secrets) or use a supported remote state
backend to store the state files in a shared location such as
S3,
Consul,
or etcd. The problem is that none of these options provide
locking, so if two team members run terraform apply
on the same state files at the same time, they may overwrite
each other's changes. The official solution to this problem is to use Hashicorp's
Atlas, but that requires using a SaaS platform for all Terraform operations and
can cost a lot of money.
The goal of Terragrunt is to provide simple locking mechanisms that are free or very inexpensive so that multiple people can safely collaborate on Terraform state.
- Install Terraform.
- Install jq.
- Install Terragrunt by going to the Releases Page, downloading the script, and adding it to your PATH.
Go into the folder with your Terraform templates and create a .terragrunt
file. This file uses the same
HCL syntax as Terraform and is used to configure Terragrunt and tell it how to do
locking.
If you want to use Git for locking (see Locking using Git), .terragrunt
should have the
following contents:
lockType = "git"
stateFileId = "my-app"
If you wish to use DynamoDB for locking (see Locking using DynamoDB), .terragrunt
should
have the following contents:
lockType = "dynamodb"
stateFileId = "my-app"
awsRegion = "us-west-2"
Now everyone on your team can use Terragrunt to run all the standard Terraform commands:
terragrunt get
terragrunt plan
terragrunt apply
terragrunt output
terragrunt destroy
Terragrunt forwards most commands directly to Terraform. However, for the apply
and destroy
commands, it will first
acquire a locking using either Git or DynamoDB, as described below.
Terragrunt can use Git as a poor-man's locking solution by trying to create a file in a repo to acquire a lock (if it fails, that means someone else already has the lock) and deleting that file to release the lock. Git is a distributed version control system, which would normally make locking impossible, but we rely on the fact that most teams have a single Git server (e.g. GitHub, BitBucket) that is used as the central source of truth and can therefore be used as the single source of locking.
To use Git for locking, you must:
- Store your Terraform templates in a Git repo (always a good idea).
- Make sure the current user has permissions to read and write from the repo using
git pull
andgit push
.
For Git locking, Terragrunt supports the following settings in .terragrunt
:
lockType = "git"
stateFileId = "my-app"
remoteName = "origin"
lockType
: (Required) Must be set togit
.stateFileId
: (Required) A unique id for the state file for these Terraform templates. Many teams have more than one set of templates, and therefore more than one state file, so this setting is used to disambiguate locks for one state file from another.remoteName
: (Optional) The name of the remote repository that should be used as the central source of truth, and therefore locking. Default:origin
.
When you run terragrunt apply
or terragrunt destroy
, Terragrunt does the following:
- Clone your local checkout into a temporary directory under
/tmp/terragrunt/YOUR-REPO
. Terragrunt will make all of its changes and commits in this/tmp/terragrunt/YOUR-REPO
folder to ensure it doesn't mess up your local checkout. - Check out the
terragrunt-lock
branch in the/tmp/terragrunt/YOUR-REPO
folder, creating the branch from master if it doesn't already exist. Terragrunt does all its commits in this branch so it doesn't dirty the commit history of your other branches. - Read the
stateFileId
value from your.terragrunt
file and create a Lock File of the same name in theterragrunt-lock
branch. The Lock File contents will include useful metadata about the lock, such as who created it (e.g. your username) and when. - Try to push the Lock File to the
terragrunt-lock
branch.- If the push succeeds, it means we have a lock!
- If the push does not succeed, it means someone else has a lock. Keep retrying every 30 seconds until we get a lock.
- Run
terraform apply
orterraform destroy
. - When Terraform is done, delete the Lock File from the
terragrunt-lock
branch to release the lock. - Delete
/tmp/terragrunt/YOUR-REPO
.
Terragrunt can use Amazon's DynamoDB as a slightly more sophisticated mechanism of acquiring and releasing locks. DynamoDB supports strongly consistent reads as well as conditional writes, which are all the primitives we need for a very basic distributed lock system. It's also part of AWS's free tier, and given the tiny amount of data we are working with and the relatively small number of times per day you're likely to run Terraform, it should be a free option for teams already using AWS.
To use DynamoDB for locking, you must:
-
Already have an AWS account.
-
Set your AWS credentials in the environment using one of the following options:
- Set your credentials as the environment variables
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
. - Run
aws configure
and fill in the details it asks for. - Run Terragrunt on an EC2 instance with an IAM Role.
- Set your credentials as the environment variables
-
Your AWS user must have an IAM policy granting all DynamoDB actions (
dynamodb:*
) on the tableterragrunt_lock_table
(see the DynamoDB locking configuration for how to configure this table name). Here is an example IAM policy that grants the necessary permissions on theterragrunt_lock_table
in regionus-west-2
for an account with account id1234567890
:{ "Version": "2012-10-17", "Statement": [{ "Sid": "", "Effect": "Allow", "Action": "dynamodb:*", "Resource": "arn:aws:dynamodb:us-west-2:1234567890:table/terragrunt_lock_table" }] }
For DynamoDB locking, Terragrunt supports the following settings in .terragrunt
:
lockType = "dynamodb"
stateFileId = "my-app"
awsRegion = "us-west-2"
tableName = "terragrunt_lock_table"
lockType
: (Required) Must be set todynamodb
.stateFileId
: (Required) A unique id for the state file for these Terraform templates. Many teams have more than one set of templates, and therefore more than one state file, so this setting is used to disambiguate locks for one state file from another.awsRegion
: (Required) The AWS region to use.tableName
: (Optional) The name of the table in DynamoDB to use to store lock information. Default:terragrunt_lock_table
.
When you run terragrunt apply
or terragrunt destroy
, Terragrunt does the following:
- Create the
terragrunt_lock_table
if it doesn't already exist. - Try to write an item to the
terragrunt_lock_table
withstateFileId
equal to the id specified in your.terragrunt
file. This item will include useful metadata about the lock, such as who created it (e.g. your username) and when. - Note that the write is a conditional write that will fail if an item with the same
stateFileId
already exists.- If the write succeeds, it means we have a lock!
- If the write does not succeed, it means someone else has a lock. Keep retrying every 30 seconds until we get a lock.
- Run
terraform apply
orterraform destroy
. - When Terraform is done, delete the item from the
terragrunt_lock_table
to release the lock.
If you shut down Terragrunt (e.g. via CTRL+C
) before it releases a lock, the lock hangs around forever, and will
prevent future changes to your state files. All Terragrunt locking mechanisms contain useful metadata, such as who
acquired the lock and when, which can be useful in determining if a lock is stale and needs to be cleaned up.
To clean up old locks, you can use the release-lock
command:
terragrunt release-lock
Are you sure you want to forcibly remove the lock for stateFileId "my-app"? (y/n):
If for some reason this doesn't work, you can also clean up locks manually:
- For Git locking, delete the lock file in the
terragrunt-lock
branch in Git (see How Git locking works). - For DynamoDB, delete the lock item in the
terragrunt_lock_table
(see How DynamoDB locking works).
- CI job
- Automated tests
- Implement best-practices in Terragrunt, such as checking if all changes are committed, calling
terraform get
, callingterraform configure
, etc.