Lock remote state before a write
Terraform state is a single mutable file; two concurrent writers lose each other's resources and produce a corrupt, hard-to-recover state. Locking serialises writes.
Procedure
- Use a backend with locking: S3 + DynamoDB lock table, GCS, or Terraform Cloud. The default
localbackend has no lock. backend "s3" { bucket = "acme-tfstate" key = "prod/terraform.tfstate" region = "us-east-1" dynamodb_table = "tf-locks" encrypt = true } - Create the lock table with a
LockIDstring primary key; without it the lock write fails and apply can proceed unlocked on older providers. - Run
terraform apply(never-lock=false) so the lock is taken for the whole plan+apply. - In CI set
-lock-timeout=5mso a queued job waits for a lock rather than failing on the first attempt. - Prove the lock is real: while an apply is paused, a second
terraform planmust block withError acquiring the state lock. - Never share one state file across environments or teams; encode the boundary in the key (
prod/,staging/). - If something genuinely holds a stale lock after a crash, use the stale-lock recovery path. Do not disable locking.
- Grant the CI identity
dynamodb:PutItem/DeleteItemon the lock table only, never*.
Pitfalls
-lock=falsein a Makefile to "avoid lock errors" removes the only protection against concurrent writers.- Two workspaces pointing at the same key but different names, which do not collide as separate lock rows.
- A DynamoDB table without the
LockIDkey, so locking quietly no-ops. - Deleting the lock table during cleanup, then applying against a bucket no one can lock.
- Long-held locks from a killed job with no timeout, blocking all deploys until manually cleared.
Verification
terraform plan 2>&1 | grep 'Error acquiring the state lock' # a second concurrent run blocks
aws dynamodb scan --table-name tf-locks --max-items 1 # row appears during apply
Report: the backend, the lock table, the observed block on a concurrent run, and the CI lock timeout.