Terraform Professional: HCP Terraform Multi-Environment Workspaces, Version Upgrades and Policy as Code

The cloud block connects a working directory to HCP Terraform or Terraform Enterprise, and its workspaces argument decides whether that directory serves one workspace or a set of environment workspaces. Use name to pin exactly one workspace; use tags to map the directory to every workspace that carries the listed tags, so dev, stage and prod can all be planned from the same code.

Tag matching is all-must-match, not any-match. If the block lists app = "payments" and env = "prod", only the workspace carrying both tags is in the set; to switch between environments from one directory, list only the tags the environments share (for example app) and keep the environment-specific tag off the block. name and tags are mutually exclusive, and prefix belongs to the older remote backend — the cloud block does not accept it.

With a tag mapping, terraform workspace new payments-sandbox creates the workspace in HCP Terraform, in the block's project (or the default project if none is given), and applies the block's tags, so the new environment immediately joins the directory's set.

Filling omitted settings from environment variables

The same root configuration can be pointed at different organizations, projects and hosts by leaving settings out of the block and supplying them as environment variables. Values written in the block always win over the variables.

TF_WORKSPACE selects but never creates: the named workspace must already exist (and, with tags, be in the matched set), so a pipeline that adds an environment provisions the workspace first — for example with the tfe provider, carrying the right tags — and then selects it non-interactively. Variables prefixed TFE_ (TFE_HOSTNAME, TFE_ORGANIZATION, TFE_TOKEN) configure the tfe provider, not the cloud block. CLI workspaces, directory-per-environment layouts and Stacks are compared in the lesson on scaling projects and multiple environments.

A VCS-connected workspace runs Terraform from a working directory on one branch of its repository, and those two settings are how one repository serves several environment workspaces.

• Folder per environment — a repository with envs/dev, envs/stage and envs/prod root configurations (calling shared modules) gets one workspace per folder, each with its working directory set to its folder. The working directory is relative to the repository root and defaults to the root.
• Branch per environment — a single root configuration can be served by workspaces that track different branches: staging tracks main, production tracks release, so promotion is a merge into release and the branch history records every promotion. Each workspace tracks the repository's default branch unless a branch is set (vcs_repo.branch in tfe_workspace).

Keep the neighbouring settings apart: trigger patterns filter runs by file paths (not branches), Git-tag triggering starts runs from tags, and run triggers chain applies between workspaces. None of them assigns a branch to a workspace. Trigger filters themselves are covered with VCS-driven workflows.

When several workspaces share the same branch and directory, the code is identical, so per-environment values belong in each workspace's Terraform variables. They are edited in HCP Terraform under workspace permissions, so a person who can commit to the repository cannot change production's values. A committed terraform.tfvars, a set of *.auto.tfvars files (every one in the directory is loaded by every workspace) or a locals map keyed by environment all live in the repository and can be changed by any committer.

Workspaces exchange data through outputs, controlled by the source workspace's remote state sharing setting, and they exchange timing through run triggers.

Remote state sharing scopes

New workspaces default to specific workspaces with an empty list, so a consumer's terraform_remote_state fails with an access error until it is added. Workspaces created before April 2021 kept the older default of organization-wide sharing, which is why audits find old workspaces wide open; tightening them means listing the actual readers (project scope would cut off any reader in another project). Remote state sharing is a workspace setting about runs reading state — granting a team read-state permission does not give another workspace's runs access.

tfe_outputs instead of terraform_remote_state

HashiCorp recommends the tfe provider's tfe_outputs data source for cross-workspace data because it returns only the source workspace's outputs, without access to its full state (terraform_remote_state works but depends on full state access). Its values attribute is marked sensitive as a whole, preemptively, so anything derived from it is sensitive — and Terraform rejects sensitive values in for_each, because they would appear in resource addresses. Use nonsensitive_values, which holds only the outputs known to be non-sensitive, for for_each keys. A variable set is not a substitute: it holds static values someone typed and does not follow the source's outputs.

Run triggers chain environments

A run trigger is configured on the downstream workspace (the one that should run), naming up to 20 source workspaces in the same organization; a successful apply in a source queues a run downstream. It carries no values — pair it with tfe_outputs for the data. Creating one requires admin access to the downstream workspace and permission to read runs on each source.

Runs queued by run triggers do not auto-apply by default. The separate setting Auto-apply run triggers (auto_apply_run_trigger in tfe_workspace) controls them independently of the workspace's main Auto-apply, which is why a workspace can auto-apply commit runs while triggered runs wait at Needs Confirmation. Trigger patterns, by contrast, react to VCS file changes, not to other workspaces.

In Local execution mode, plans and applies run on the operator's machine and HCP Terraform acts only as the state backend: it stores each new state version and locks the workspace while a local operation runs, so two people cannot write state at once.

Everything that belongs to a remote run stops:

• HCP Terraform evaluates no workspace variables and no variable sets — not even project- or organization-owned sets — so credentials and inputs must come from the local machine (tfvars files, environment variables). A plan that suddenly prompts for a variable or fails to authenticate after switching modes is this behaviour;
• No policy sets are evaluated, and local plans and applies do not appear in the workspace's run list.
• The CLI-versus-workspace Terraform version check (see the upgrade section) is skipped.

Switching back to Remote (or Agent) mode restores variables, policies and run history. Because the switch changes how every run executes, it is a heavy tool for a one-off problem.

Each HCP Terraform workspace has a Terraform version setting that chooses the binary its runs use; it can be an exact version or a version constraint.

• Default: the release that was current when the workspace was created. It never moves by itself, so an organization accumulates different versions by workspace age; upgrading is a deliberate change.
• Exact version ("1.12.4"): fixed until someone edits it — right for production, where a newly published patch must not change runs without review.
• Constraint: the workspace uses the newest available release that satisfies it, and moves when a new matching release appears. "~> 1.12.0" takes every 1.12 patch and never 1.13; "~> 1.12.4" still admits 1.12.5 and later; ">= 1.12.0" admits new minor releases too.

The state file's recorded version does not choose the binary, and the configuration's required_version only checks the running binary — it never sets the workspace version (its semantics are covered with version constraints). Manage the setting as code with tfe_workspace's terraform_version: with one resource using for_each over an environment map, put a version in each map entry so dev can move first while stage and prod stay put, every move being a reviewed pull request. Setting one value for all and applying with -target leaves code and reality disagreeing; a UI edit bypasses review.

Terraform Enterprise: site-admin Terraform versions

On Terraform Enterprise, site administrators maintain the list of Terraform versions workspaces can choose. New releases are not added automatically, which matters for air-gapped installations: an admin adds a version with its number, a Linux 64-bit download URL (an internal mirror when air-gapped) and the binary's SHA256 checksum. Status controls availability:

The list shows how many workspaces use each version, so retiring an old release means moving those workspaces first; a version cannot be disabled while any workspace still selects it. Don't edit the default version entries (they are overwritten) — add a new version instead.

A safe upgrade proves the configuration on the new version before any run can apply with it, then rewrites state with the new version. Per workspace, the order is:

1. Widen required_version so the configuration admits the new release (with "~> 1.12.0" in the code, every 1.13 run of any kind fails). While the workspace still runs its old release, this changes nothing else.
2. Run a plan-only (speculative) run on the new version. A plan-only run can select a Terraform version for that run alone; the workspace keeps its setting for every other run, and a speculative plan can never be applied.
3. Change the workspace's Terraform version (HashiCorp's tutorial changes both the workspace version and the configuration constraint).
4. Apply an empty run. State records the version that last wrote it, and HCP Terraform does not apply a no-change run by default, so the Allow empty apply run type (allow-empty-apply in the Runs API) applies with no resource changes and writes state with the new version. A speculative plan never writes state.

Runs API rules worth knowing: terraform-version is valid only on plan-only runs (not on auto-apply or saved-plan runs), so a version override can never apply changes; allow-empty-apply runs on the workspace's own version; refresh-only cannot be combined with is-destroy. Staged rollout strategy and state-downgrade risk are covered with version management.

The CLI-versus-workspace version check

When the local CLI touches a remote-execution workspace's state (terraform state mv, state rm, import and similar), the cloud backend compares the CLI version with the workspace's version setting — not with the release that setting resolves to.

The error reads like "The local Terraform version (…) does not meet the version requirements for remote workspace …". The safe fix is to run the command with a CLI in the accepted series. -ignore-remote-version downgrades the error to a warning, but a newer CLI may then write state the workspace's own runs cannot read. Local execution mode also skips the check, but it changes how every run executes.

sentinel test runs every test case it finds at test/<policy>/*.hcl (or *.json), where <policy> is the policy file's name without .sentinel; each file is one case. With that layout a CI job runs sentinel test with no arguments. (tests/*.tftest.hcl belongs to terraform test, and sentinel.hcl configures the policy set, not test cases.)

A test case file can contain:

• mock blocks — replace an import such as tfplan/v2, tfconfig/v2, tfstate/v2 or tfrun with mock data (inline or from a file via source);
• param and global blocks — supply parameter values and inject values into the policy's scope;
• a test block — the expected results, e.g. test { rules = { main = false } }.

A case with no test block expects the policy to pass. A "fail" case built to be denied therefore reports FAIL even when the policy works, until it asserts main = false. Enforcement levels are policy set configuration and do not change a test's expectation.

Mock data from real runs

HCP Terraform generates Sentinel mocks from a run's plan, which gives realistic test data:

• UI: on the run page, expand the plan section and choose Download Sentinel mocks. The bundle holds mock files and a sample sentinel.hcl for sentinel apply.
• API: create a plan export of type sentinel-mock-bundle-v0, poll until it finishes, and download it within about one hour, after which it expires.
• Only a successful plan is required — a run that later failed a policy or an apply still offers mocks, which makes a failing run the ideal source for a reproduction case.
• The caller needs the Download Sentinel mocks permission on the workspace; reading runs is not enough.
• HCP Terraform attempts to scrub sensitive values but does not guarantee it, so review mock files before committing them.

Store mocks in a separate testdata directory so policy detection does not pick them up. Mind the two path bases: the sample sentinel.hcl uses paths relative to the working directory (so sentinel apply works from the repository root), but paths inside a test case are relative to the test file. A case at test/<policy>/<case>.hcl sits two levels below the repository root, so it reaches the root's testdata/ by climbing two levels first.

HCP Terraform OPA policies are ordinary Rego, so they are unit-tested with opa test: rules named test_… evaluate the policy with mocked input (with input as …), and nothing contacts HCP Terraform. sentinel test cannot evaluate Rego, and opa eval against a live speculative plan would need HCP Terraform.

The mock must have the shape HCP Terraform builds at evaluation time, which combines two documents:

A rule about a run or workspace property reads it under input.run (a rule that blocks destroy runs reads input.run.is_destroy), so its test mocks that path; a rule about resources reads input.plan.resource_changes.

Building input from a real plan

The JSON execution plan is downloadable through GET /api/v2/plans/:id/json-output or /runs/:id/plan/json-output. These endpoints:

• reject organization tokens and need a user or team token with admin access to the workspace (read-runs is not enough);
• answer with a temporary redirect (HTTP 307) to a URL valid for about a minute — request a fresh one for each download;
• return the raw Terraform plan, sensitive values included, without the plan/run wrapper. (The sanitized plan is a separate endpoint for HYOK workspaces.)

Passed straight in as input, the file has no input.plan, so a correct policy returns an empty deny. Wrap it as {"plan": <plan JSON>} (adding a run object if the policy needs one). Never "fix" the test by rewriting the policy to read input.resource_changes — that breaks it in real runs.

A policy set takes its policies from one source: a connected VCS repository, or versions uploaded through the API (directly or via the tfe provider). The choice decides how a pipeline ships tested policies.

In a monorepo, a VCS-connected set uses policies_path to load only its folder and policy_update_patterns so application commits don't create policy versions. working_directory and trigger_patterns are the workspace equivalents and are not policy set arguments; policies_path and policy_update_patterns do nothing without a VCS connection. A slug cannot be combined with a VCS connection, and the API refuses uploads to a VCS-connected set with a 422 ("The policy set does not support uploading versions") — changes reach it only by merging into the tracked branch. Configuration versions are the workspace counterpart and carry Terraform code, not policies; policy_ids builds a set from individually managed tfe_policy resources instead of uploading a directory.

The Sentinel or OPA release that evaluates a policy set is a policy set setting, policy_tool_version, not something inherited from the workspace or its agent. Pinning it makes policy results reproducible between CI tests and real runs.

Sentinel

An unset version defaults to latest (the Policy Sets API default; the tfe provider selects the latest available version when it is left out), so the runtime is not pinned to a release you chose; HashiCorp recommends choosing a specific version. Scoping (global, project or workspace selection) and mock generation do not depend on the mode. Enforcement levels, scoping and overrides themselves are covered with policy enforcement.

OPA

• OPA sets run in Agent mode by default (agent_enabled true).
• policy_tool_version = "latest" is rejected for OPA sets; according to the tfe provider documentation and the Terraform Enterprise release note that deprecated OPA latest, sets that were created on latest were pinned to the most recent supported release.
• The runtime changes only when policy_tool_version is set to a newer explicit version — uploading a new policy set version changes the policies, not the runtime.