Terraform State: Backends, Locking, Remote State, and Drift

The local backend is what Terraform uses when your configuration declares no backend or cloud block: it stores state as a terraform.tfstate file in the current working directory and keeps the previous version in terraform.tfstate.backup. You never have to configure it - run terraform init and terraform apply in a fresh directory and the local backend is already in effect.

You can also declare it explicitly, which is mainly useful for pointing state at a non-default path:

terraform {
  backend "local" {
    path = "state/terraform.tfstate"
  }
}

Do not confuse the state file with the .terraform/ directory. .terraform/ holds the downloaded providers, installed modules, and a cached copy of the backend settings from the last terraform init; it does not hold your local state, which sits beside it in the working directory.

The local backend does support state locking - Terraform takes a lock on the state file on disk, so two terminals on the same machine cannot write it simultaneously. What it cannot do is share state. The file exists only on one machine, and it should not be committed to version control (it holds secrets in plain text, and Git gives neither locking nor safe merging). So a teammate who clones the repository gets no state at all: their terraform plan believes nothing exists yet and proposes creating every resource again. That is exactly why a team or production workflow needs a remote backend.

State locking makes Terraform acquire a lock before any operation that could write state and hold it until that operation finishes. Without it, two people running terraform apply against the same state at the same time could interleave writes and leave the file corrupted or missing resources. Locking happens automatically whenever the backend supports it; there is no configuration setting that enables it for the run.

Which commands lock? apply, destroy, import, the state-modifying terraform state subcommands - and plan, which holds the lock for its whole run, with or without -refresh=false. Reading another configuration's outputs with the terraform_remote_state data source is a read-only fetch and does not take that state's lock.

When another run already holds the lock, what happens depends on two per-command flags:

A lock left behind by a crashed run never expires on its own. The recovery tool is terraform force-unlock LOCK_ID:

terraform force-unlock 6f2b1c3e-1a2b-4c5d-8e9f-0123456789ab
terraform force-unlock -force 6f2b1c3e-1a2b-4c5d-8e9f-0123456789ab   # skip the confirmation prompt

• It takes the lock ID from the error message and asks for confirmation unless you pass -force.
• It removes the lock for the current configuration and nothing else: it performs no check that a run is still active, and it does not repair state that a crashed apply left half-written.
• Local state locks cannot be unlocked by another process - on the local backend, stop the process that holds the lock instead.

Use it only when you are certain no operation is still running; force-unlocking a live run reopens the door to corruption.

Most widely used backends lock natively, but each does it with its own storage primitive:

The s3 backend is the one with traps. use_lockfile defaults to false, so deleting dynamodb_table without adding use_lockfile = true leaves the backend with no locking at all. The lock object needs s3:GetObject, s3:PutObject, and s3:DeleteObject permission on <key>.tflock in addition to the permissions on the state object. During a migration you may set dynamodb_table and use_lockfile = true together: Terraform then acquires both locks, and you remove the DynamoDB setting once every user and pipeline runs a version that supports the S3 lock file.

terraform {
  backend "s3" {
    bucket         = "acme-tf-state-prod"
    key            = "network/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    use_lockfile   = true
    dynamodb_table = "tf-locks"   # transitional; deprecated - remove after migrating
  }
}

You configure remote state by adding a backend block inside the top-level terraform block, naming a backend type and its settings. A configuration can have at most one backend, and cannot combine a backend block with a cloud block (the HCP Terraform integration). Built-in backends include s3, gcs, azurerm, kubernetes, consul, pg, http, oss, cos, and remote.

terraform {
  backend "s3" {
    bucket       = "acme-tf-state-prod"
    key          = "network/terraform.tfstate"
    region       = "us-east-1"
    encrypt      = true
    use_lockfile = true
  }
}

A Google Cloud equivalent uses the gcs backend, which locks natively with no extra settings:

terraform {
  backend "gcs" {
    bucket = "acme-tf-state"
    prefix = "prod/network"
  }
}

Two rules are heavily tested. First, the backend is initialised before the rest of the configuration is evaluated, so backend arguments cannot reference any named value - var., local., data source attributes, and even terraform.workspace are all rejected. Every argument must be a literal, or be supplied at init time (next section). For the same reason, TF_VAR_ environment variables and .tfvars files cannot feed a backend: they set input variables, which the backend block cannot read. Second, any change to the backend block only takes effect after you re-run terraform init.

Remote state gives the team one shared source of truth with cross-machine locking and durable storage - but the state still contains sensitive values. Marking an output or variable sensitive = true only redacts it from CLI output; the value is still written to state in plain text. Protect state where it lives: tight access control on the bucket or container, and encryption at rest (for example encrypt = true on s3). The one way to keep a secret out of state entirely is a write-only argument fed an ephemeral value (Terraform 1.11+), where the provider supports it.

Partial configuration means leaving some backend arguments out of the backend block and supplying them when you run terraform init. Because the block cannot use variables, this is the supported way to vary settings per environment or keep them out of the committed .tf files. You pass the missing values with -backend-config, either as a file or as individual key/value pairs. HashiCorp recommends naming the files *.backendname.tfbackend:

terraform init -backend-config=prod.s3.tfbackend

terraform init \
  -backend-config="bucket=acme-tf-state-prod" \
  -backend-config="key=network/terraform.tfstate" \
  -backend-config="region=us-east-1"

The rules behind it:

• Only init takes backend settings. plan and apply have no -backend-config option; they use what init cached in .terraform/.
• Order matters. -backend-config options are processed in the order given, so a later one wins over an earlier one, and all of them override values in the block.
• Missing values prompt - unless input is off. Terraform asks interactively for required arguments that are still missing; with -input=false (typical in CI) init fails instead.
• Not a secret store. Values passed with -backend-config (or hardcoded in the block) are written in plain text to .terraform/terraform.tfstate and into saved plan files. Supply credentials through environment variables or credential files the backend reads directly (for s3, the standard AWS credential chain) instead.

When you change the backend - say, moving from local to s3 - terraform init detects the change and will not proceed until you choose what to do with the existing state:

There is no -auto-approve flag on init. Migration is the normal path: it is how you promote a laptop-only project to shared remote state without losing the mapping to existing infrastructure.

The terraform_remote_state data source lets one configuration read another configuration's latest state snapshot and consume its root module outputs. It is how a separately managed application stack can reference, for example, the subnet IDs that a network stack created, without hard-coding them:

data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "acme-tf-state-prod"
    key    = "network/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_instance" "app" {
  ami           = "ami-0abcd1234example"
  instance_type = "t3.micro"
  subnet_id     = data.terraform_remote_state.network.outputs.subnet_id
}

Values are always reached through data.terraform_remote_state.NAME.outputs.OUTPUT. Only values the producing configuration declares as output blocks in its root module are visible - resources, and child-module outputs that the root did not re-export, are not reachable. The read does not lock the producer's state.

The catch is access. Although only outputs are exposed, the consumer must be able to read the entire state snapshot, which can include sensitive values. The optional defaults and workspace arguments do not narrow that, and storage permissions cannot either: S3 grants access to whole objects, so neither a bucket policy nor reading the object another way can expose just the outputs. When the consumer must not read the producer's state, HashiCorp recommends publishing the values explicitly instead:

• Have the producer write them to a separate store - for example AWS SSM Parameter Store, Consul, or a DNS record - and read them with that provider's ordinary data source.
• On HCP Terraform, use the tfe_outputs data source, which reads a workspace's outputs without access to its full state.

Use terraform_remote_state when both sides are Terraform, you control both, and the consumer is trusted with the producer's state.

Drift is when real infrastructure no longer matches what Terraform state recorded because something changed it outside Terraform - someone resized an instance in the console, deleted a security rule by hand, or an external process changed a tag. Terraform detects drift by refreshing: reading each remote object recorded in state through its provider and comparing it with state. Refresh only reads objects that are already in state, so a resource created by hand that Terraform never managed is not drift and is never reported; bringing it under management is a job for import (objective 7).

A normal terraform plan refreshes first, then diffs the refreshed picture against your configuration. The refreshed values live in memory for that plan only - plan does not persist them to the state file. How drift shows up depends on what changed:

Three planning modes treat drift differently:

The standalone terraform refresh command still exists but is deprecated: it behaves like terraform apply -refresh-only -auto-approve, silently rewriting state with no chance to review.

Once drift is visible you have two honest options. To undo it, run a normal terraform apply to push infrastructure back to the configuration. To keep it, change the configuration to match and, if you want state updated first, run terraform apply -refresh-only. Accepting drift into state is not enough on its own: a refresh-only apply updates state but not the configuration, so with the configuration unchanged the next normal plan proposes reverting the value again.

The terraform state subcommands (plus a few related top-level commands) inspect and rewrite the state's mapping without touching infrastructure. They belong mainly to objective 7 (using the CLI to inspect state, covered in the lesson on import, state inspection, and debug logging); this summary shows where they fit next to backends and drift:

The deprecation pattern recurs: terraform taint gave way to the plannable terraform apply -replace="aws_instance.web", just as terraform refresh gave way to -refresh-only. Configuration-driven alternatives exist for state surgery too - a moved block records a rename in configuration instead of a one-off state mv, and a removed block (Terraform 1.7+) forgets a resource in a reviewed plan instead of state rm.

terraform state mv aws_instance.app aws_instance.web
terraform apply -replace="aws_instance.web"

Remember the boundary: state mv and state rm edit only Terraform's records. state rm never destroys anything - the resource keeps running, Terraform just stops managing it (and a later plan may propose creating a duplicate if the configuration still declares it).