Terraform Professional (002): Dynamic Provider Credentials and Variable Files

Dynamic provider credentials let an HCP Terraform or Terraform Enterprise run authenticate to a cloud with short-lived credentials created for that run, instead of static keys stored as workspace variables. Nothing long-lived is stored, so there is nothing to rotate or leak.

The flow has four steps:

1. For each run phase (plan and apply), HCP Terraform signs a workload identity token — an OIDC-compliant JWT — describing the run: which organization, project and workspace it belongs to and which phase it is in.
2. The cloud (or Vault) has been configured to trust HCP Terraform as an identity provider: an IAM OIDC identity provider in AWS, a federated identity credential on a Microsoft Entra ID app registration, a workload identity pool provider in Google Cloud, or a JWT auth method in Vault. The issuer is the HCP Terraform address, https://app.terraform.io, or your Terraform Enterprise hostname.
3. The trust configuration checks the token's claims — at minimum the audience and the organization, usually also the project, workspace and phase — and, if they match, exchanges the token for temporary credentials.
4. HCP Terraform hands those credentials to the provider outside your configuration (for AWS, as a generated shared config file), so the provider block needs no authentication arguments.

Two places therefore have to agree for a run to authenticate: the workspace variables that tell HCP Terraform to request a token and which identity to use, and the trust conditions in the cloud that decide which tokens may use that identity. Most troubleshooting comes down to telling these apart. If no token exchange was even attempted (for AWS, no AssumeRoleWithWebIdentity call in CloudTrail), the workspace side is wrong; if the exchange was attempted and denied, the trust side is wrong.

Native integrations include AWS, Microsoft Azure, Google Cloud, Vault, and Kubernetes and Helm, plus Vault-backed variants for AWS, Azure and Google Cloud (covered below). Securing the pipeline identities that run outside HCP Terraform, such as a GitHub Actions role, is part of the security domain; this lesson covers the HCP Terraform side.

AWS dynamic credentials are enabled with two environment variables on the workspace: TFC_AWS_PROVIDER_AUTH=true and TFC_AWS_RUN_ROLE_ARN set to the IAM role to assume. The role must trust an IAM OIDC identity provider for the HCP Terraform address whose audience list contains the token's audience (aws.workload.identity by default).

Three details trip up migrations:

• Category. These settings are read from the run's environment, so they must be in the environment variable category. Entered as Terraform variables, they become values for input variables and HCP Terraform ignores them for authentication. The category is the only setting that decides how HCP Terraform reads the variable.
• HCP Terraform's names, not the AWS SDK's. HCP Terraform creates and places the token itself. SDK variables such as AWS_ROLE_ARN or AWS_WEB_IDENTITY_TOKEN_FILE are not how you configure it, and no assume_role_with_web_identity block is needed in the provider.
• Leftover static keys win. The AWS provider looks for credentials in a fixed order: provider arguments, then environment variables such as AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY, then shared config and credentials files. Dynamic credentials arrive as a shared config file, so old access-key variables still on the workspace (or in a variable set) silently take precedence, and CloudTrail shows the old IAM user instead of the role. Delete them; HashiCorp warns against combining other authentication methods with dynamic credentials.

With a single configuration the provider block can be as small as provider "aws" { region = "us-east-2" }; HCP Terraform supplies the credentials implicitly.

The trust policy decides which runs may use a role, and it does so by matching claims in the workload identity token. The most important claim is sub, which always has this fixed order:

organization:<ORG>:project:<PROJECT>:workspace:<WORKSPACE>:run_phase:<plan|apply>

The token also carries aud (the audience), the issuer, and individual claims such as terraform_organization_name, terraform_project_name, terraform_workspace_name and terraform_run_phase, plus ID claims (terraform_organization_id, terraform_project_id, terraform_workspace_id). An AWS trust policy for one workspace, both phases, looks like this:

"Condition": {
  "StringEquals": { "app.terraform.io:aud": "aws.workload.identity" },
  "StringLike":   { "app.terraform.io:sub":
    "organization:example-org:project:billing:workspace:billing-api:run_phase:*" }
}

• StringLike vs StringEquals. Only StringLike evaluates * as a wildcard. Under StringEquals the asterisk is literal, so a pattern matches no real token and every run is refused.
• Scope by wildcarding segments. Wildcard only the phase to admit one workspace; wildcard the workspace segment (workspace:*) to admit every current and future workspace in a project; use a name pattern (workspace:dns-*) to separate teams that share a project. HashiCorp's guidance is to always check at least the audience and the organization name — a project or workspace name alone is not unique across organizations.
• The audience lives in aud, the path lives in sub. Neither appears in the other.

When the claims change

Tokens are built from the names that exist at run time. Renaming an organization, project or workspace, or moving a workspace to another project, changes sub immediately, and every trust condition that matched the old name starts denying the exchange (an AccessDenied on sts:AssumeRoleWithWebIdentity). The issuer and audience do not change with a rename. Plan renames and moves together with trust-policy updates.

The ID claims are the durable alternative: an organization, project or workspace keeps its ID when renamed. A trust system that can match custom claims (a Google Cloud attribute mapping and condition, or a Vault JWT role's bound claims) can pin IDs instead of names and survive renames. An AWS IAM trust policy can only test the standard OIDC condition keys such as sub and aud, so on AWS the name-based sub remains what you match.

A custom audience (TFC_AWS_WORKLOAD_IDENTITY_AUDIENCE) changes the token's aud, which AWS checks in two places: STS accepts the token only if its aud is in the IAM OIDC identity provider's audience list, and the role's trust policy compares aud in its condition. Both must be updated to the new value.

Tokens for tools without a native integration

For a custom workflow such as an agent hook that must authenticate to an internal system, set TFC_WORKLOAD_IDENTITY_AUDIENCE to the audience that system expects. HCP Terraform then generates a token with that audience and exposes it to the run as TFC_WORKLOAD_IDENTITY_TOKEN. Each integration-specific audience variable shapes only the token for that integration.

Separate plan and apply identities let a workspace plan with a read-only role and apply with a write role. HCP Terraform issues a different token for each phase (run_phase:plan and run_phase:apply), so each role can trust only its own phase.

The phase-specific variables always win, and each falls back to the run role only when it is missing:

The rule behind the table: a phase-specific value is read first and the run role is consulted only for a phase that has none. Different combinations can therefore produce the same outcome.

Each role's trust policy must match the phase it serves. A frequent failure is creating both trust policies from one template: if the apply role still trusts run_phase:plan, plans succeed and every apply is denied. The fix is on the apply role's trust condition, not in the variable names. Naming matters: the phase is written inside the variable name (TFC_AWS_PLAN_ROLE_ARN), while anything appended at the end is a tag that defines a separate configuration (see the multiple-configurations section).

Azure and Google Cloud follow the same token-and-trust model as AWS, with their own variable names and one important difference in how matching works.

On Azure, an app registration with only a run_phase:plan credential lets plans through and refuses applies; the fix is a second credential with a run_phase:apply subject. Wildcards are not evaluated in a standard credential's subject. (Entra's newer flexible federated identity credentials are a separate feature; the standard credential is exact-match.)

On Google Cloud, write conditions on assertion.sub so the organization is pinned along with the project; a condition on terraform_project_name alone would admit a same-named project in another organization. TFC_GCP_PRINCIPAL_TYPE defaults to service_account, which is what a run service account email needs. Remove any GOOGLE_CREDENTIALS or GOOGLE_APPLICATION_CREDENTIALS variable left from a key-file setup — HashiCorp warns they conflict with dynamic credentials.

One workspace can hold several configurations for the same provider — for example a default AWS account plus a central logging account used by an aliased provider. Each extra configuration is defined by appending a tag to the variable names, and each must be switched on with its own variable:

TFC_AWS_PROVIDER_AUTH=true                 # default (untagged) configuration
TFC_AWS_RUN_ROLE_ARN=arn:aws:iam::123456789012:role/app
TFC_AWS_PROVIDER_AUTH_LOGGING=true         # tagged configuration "LOGGING"
TFC_AWS_RUN_ROLE_ARN_LOGGING=arn:aws:iam::210987654321:role/logs

HCP Terraform passes the credentials for every configuration in one root-module variable that you declare, tfc_aws_dynamic_credentials: default holds the untagged configuration and aliases is a map keyed by tag. For AWS each entry has a shared_config_file:

variable "tfc_aws_dynamic_credentials" {
  type = object({
    default = object({ shared_config_file = string })
    aliases = map(object({ shared_config_file = string }))
  })
}

provider "aws" {
  shared_config_files = [var.tfc_aws_dynamic_credentials.default.shared_config_file]
}

provider "aws" {
  alias               = "logs"
  shared_config_files = [var.tfc_aws_dynamic_credentials.aliases["LOGGING"].shared_config_file]
}

• Map every provider by hand. With one configuration, credentials reach the provider implicitly. Once you add a second, HashiCorp requires mapping authentication for all providers, including the unaliased default one — otherwise the default provider stops authenticating.
• The key is the variable tag, not the provider alias. HCP Terraform never sees the alias label in your HCL; aliases["LOGGING"] comes from the _LOGGING suffix, even though the provider alias is logs.
• Per-provider object shapes differ. Azure's object carries client_id and oidc_token_file_path (in tfc_azure_dynamic_credentials); Google Cloud's carries credentials for the provider. Each provider has its own tfc_<provider>_dynamic_credentials variable.

Shared defaults with TFC_DEFAULT_

Most dynamic-credential variables have a TFC_DEFAULT_ counterpart (for example TFC_DEFAULT_AWS_RUN_ROLE_ARN) whose value applies to every configuration that does not set its own. An explicit value — tagged or untagged — always wins over the default. The exception is the switch: TFC_AWS_PROVIDER_AUTH (and its Azure/GCP/Vault equivalents) has no default form, so TFC_DEFAULT_AWS_PROVIDER_AUTH enables nothing. Every tagged configuration needs its own TFC_AWS_PROVIDER_AUTH_<TAG>=true, and the untagged switch enables only the default configuration.

Credentials are delivered across many workspaces with variable sets holding the TFC_ environment variables, matched on the cloud side by trust conditions with the same scope. Getting the two halves to line up is the core of objective 4c.

Which value a run receives follows HCP Terraform's precedence:

• Workspace variables override variable sets with the same key — the least-effort way to give one workspace a different role while it stays in the project.
• Among non-priority sets, the more specific scope wins: sets applied to the workspace and project-scoped sets outrank global sets. Lexical order of set names only breaks ties between sets at the same rank.
• Ownership also counts. Variable sets can be owned by the organization or by a project, and HashiCorp's full list ranks them by owner as well as scope — for example a project-owned set scoped to the project ranks above an organization-owned set applied to the workspace. Check the current precedence table when sets of both owners overlap.
• A priority variable set overrides everything at more specific scopes, including workspace variables and command-line values; it is a guardrail for values that must not be changed, not a tool for per-workspace exceptions.
• A TFC_DEFAULT_ value never overrides an explicit value from a set; it only fills gaps.

Scoping traps to avoid: a global set reaches every workspace in the organization (too wide for a project requirement); per-workspace variables do not reach workspaces created later; and moving a workspace to another project to pick up a different set also changes the project segment of its sub claim, so the role's trust may stop matching. A role ARN is not a secret: the trust policy, not the variable, controls who can use the role.

Vault-backed dynamic credentials route authentication through Vault: the run authenticates to Vault with its workload identity token, and Vault's AWS, Azure or Google Cloud secrets engine issues the cloud credentials. The cloud never validates an HCP Terraform token; Vault does.

Reasons to choose it over direct dynamic credentials: credential issuing and auditing stay centralised in Vault; and the cloud does not need to reach HCP Terraform's OIDC metadata, which matters for a Terraform Enterprise instance on a private network that the cloud provider cannot reach.

Setup is layered:

1. Vault dynamic credentials first — a JWT auth method and role in Vault trusting HCP Terraform's tokens, plus TFC_VAULT_PROVIDER_AUTH=true, TFC_VAULT_ADDR and TFC_VAULT_RUN_ROLE (and TFC_VAULT_NAMESPACE if used).
2. Then the Vault-backed variables, e.g. for AWS: TFC_VAULT_BACKED_AWS_AUTH=true, TFC_VAULT_BACKED_AWS_AUTH_TYPE (iam_user, assumed_role or federation_token), TFC_VAULT_BACKED_AWS_RUN_VAULT_ROLE (the role in the secrets engine) and, optionally, TFC_VAULT_BACKED_AWS_MOUNT_PATH, which defaults to aws.
3. For assumed_role, also TFC_VAULT_BACKED_AWS_RUN_ROLE_ARN — the AWS role Vault should assume — unless both the PLAN_ROLE_ARN and APPLY_ROLE_ARN variants are set. It matters especially when the Vault role lists more than one role ARN.

The direct TFC_AWS_PROVIDER_AUTH/TFC_AWS_RUN_ROLE_ARN variables belong to the other integration and are not read here. When migrating from code that reads keys with a data source such as vault_aws_access_credentials and passes them to access_key/secret_key, HashiCorp requires removing that secrets-engine usage and the provider's own authentication arguments; data source results are written to state even when values are marked sensitive. The aim is that no Vault token or cloud key is stored in the workspace at all: the workload identity token is the only credential the run starts with.

Terraform CLI resolves an input variable from several sources and keeps the last value found, in this order from lowest to highest:

1. The default in the variable block (used only when nothing sets it)
2. Environment variables TF_VAR_<name>
3. terraform.tfvars
4. terraform.tfvars.json
5. *.auto.tfvars and *.auto.tfvars.json, together, in lexical order of file name
6. -var and -var-file options, in the order given on the command line

What this means in practice:

• Command-line position decides, not flag type. -var has no rank over -var-file. Whichever option that sets the variable comes later on the line wins, whether it is a flag or a file: -var="node_pool_size=4" -var-file=region-b.tfvars takes the file's value, and swapping their order takes the flag's. Layering -var-file=common.tfvars -var-file=prod.tfvars is the usual pipeline pattern for environment overrides.
• Among auto files, only the name matters. net.auto.tfvars beats base-config.auto.tfvars.json because "n" sorts after "b"; the .json form only ranks terraform.tfvars.json above terraform.tfvars.
• Auto-loading is root-directory only. terraform.tfvars, terraform.tfvars.json and *.auto.tfvars(.json) are read from the root module's working directory; files in subdirectories are never auto-loaded. A file without .auto (such as prod.tfvars) loads only when named with -var-file. A pipeline that needs a per-run override can write a generated ci-values.auto.tfvars into the root directory or pass a generated file with -var-file.
• TF_VAR_ cannot beat a file. Environment variables rank below every variable file, so exporting one does not override a committed terraform.tfvars.
• No errors for layering. The same variable in different files is resolved by precedence without a warning; assigning it twice inside one file is an error.

Maps and objects are replaced whole

Since Terraform 0.12, map and object values behave like any other value: a higher-precedence source replaces the whole value; nothing is merged across files. If terraform.tfvars sets labels = { team = "data" } and region.auto.tfvars sets labels = { tier = "gold" }, the result has only tier. For an object type with optional(type, default) attributes, the winning value is taken whole and then the declared defaults fill the attributes it omits — they never pull values from the losing file. To layer shared and per-environment keys, declare two variables and combine them in configuration:

locals {
  labels = merge(var.common_labels, var.env_labels)
}

Basic variable declaration and type handling are covered in the variables lesson of the configuration-design domain.

In HCP Terraform, values from the workspace and from variable sets join the file and command-line sources. HashiCorp's precedence, from highest to lowest:

1. Priority variable sets (they override same-key values at every more specific scope, including command-line flags)
2. Command-line -var / -var-file values (CLI-driven runs only)
3. Local TF_VAR_ environment variables (CLI-driven runs only)
4. Workspace-specific variables
5. Variable sets applied to the workspace, then project-scoped sets, then global sets (ranked also by whether the organization or a project owns the set)
6. *.auto.tfvars files in the configuration
7. terraform.tfvars in the configuration

Consequences worth knowing:

• Workspace variables beat committed files. A workspace variable overrides a value in the repository's *.auto.tfvars or terraform.tfvars; edit time plays no part.
• VCS-driven workspaces need per-environment values in workspace variables. Nobody passes -var-file in a VCS-driven run, so a committed prod.tfvars is never read; committed *.auto.tfvars files load in every workspace that uses the branch, so dev.auto.tfvars and qa.auto.tfvars would both load and lexical order would pick one for both environments.
• A priority set beats even -var. Only changing the configuration itself bypasses it, which is why HashiCorp pairs priority sets with policy checks where that matters. Priority sets in project design are covered in the workspaces and environments lesson.
• JSON variable files. HashiCorp's HCP Terraform list names *.auto.tfvars and terraform.tfvars; the JSON forms follow the same file rules as the CLI.
• Terraform Enterprise and terraform.tfvars. HashiCorp's variables documentation currently states that Terraform Enterprise ignores terraform.tfvars, while HCP Terraform reads it. A value that must hold on both products belongs in a *.auto.tfvars file or a workspace variable; otherwise a migrated workspace silently falls back to the variable's default.