Terraform Professional (002): Policy Rules, Overrides and Risky Changes

Policy as code in HCP Terraform means writing compliance rules as Sentinel or OPA (Rego) policies, grouping them into policy sets that the organization owns, and letting HCP Terraform evaluate them on every run in the workspaces each set covers. A policy set holds policies from one framework only, and the framework cannot be changed after the set is created. (HashiCorp has also introduced a native HCL-based framework, Terraform policy, for newer Terraform versions; the governance model below applies to it too, but this lesson uses Sentinel and OPA.)

Where policies run in the run lifecycle

Policies evaluate after a successful plan and before apply. The order is plan → (cost estimation, for Sentinel policy checks) → policy stage → apply. A run whose plan errors ends in Plan Errored: it never reaches the policy stage and can never apply, so a missing policy result on an errored run does not mean the policies were bypassed.

Why a policy set, and not a rule in the code

Terraform's own guardrails — variable validation, precondition/postcondition, check blocks and lifecycle settings — are part of the configuration. Whoever controls the code controls the rule: a team that copies a module can delete its validation, and a check block only warns. A policy set is managed at organization level by teams holding Manage policies, scoped by the platform team and evaluated on every run regardless of where the configuration came from. That is what makes a rule unbypassable.

Rule of thumb: rules you can express directly over Terraform data belong in a policy set; results that come from a third-party tool arrive through a run task (set to mandatory if a failure must stop the run). A run task attached in a workspace's own settings can be removed by that workspace's admins, so it is the wrong home for a control the workspace owners must not weaken. Run task stages and setup are covered in the CI/CD lesson.

Core compliance rules go in a globally scoped policy set; team or project rules go in additional sets scoped to just those workspaces. A policy set can apply globally (every workspace in the organization, including ones created later), to specific projects, to specific workspaces or Stacks, or to workspaces that carry particular tags.

Sets add up; they never relax each other

Every policy set whose scope includes a workspace is evaluated for its runs, independently. A project-scoped set cannot weaken a global one: if the global set fails a hard-mandatory policy, a project set holding the same rule at advisory level changes nothing — the global failure still stops the run. To exempt something, change the scope of the set that is failing it (see exclusions below).

Scope is resolved from where the workspace is now

Project scope follows current membership. Move a workspace out of a project and that project's sets stop applying on its next run, while global sets keep applying; nothing is carried over and nothing needs detaching. If a rule must follow a workspace regardless of project, scope the set to the workspace (or a tag) instead.

An exclusion removes a project, workspace, Stack or tag from a policy set's scope while the rest of the scope stays as it is. It is the right tool when one part of the estate has an approved exemption from an otherwise global rule, because the set remains global and so keeps covering every new workspace and project automatically.

Pick the exclusion unit that matches the group being exempted:

• One workspace (a single demo environment) — exclude that workspace.
• A whole project whose workspaces come and go (a training project) — exclude the project, so new workspaces in it are skipped without upkeep.
• A group spread across projects that shares a tag (short-lived workspaces tagged ephemeral in several teams' projects) — exclude the tag; it is configured once and applies to future tagged workspaces.

The alternatives each break something. Rescoping a global set to a list of workspaces or projects stops covering anything created later. Per-workspace exclusions for a growing group need maintenance every time one appears. Lowering a policy to advisory relaxes it for everyone. Adding a skip condition inside the policy code (for example a when clause on workspace tags) can work but edits policy code, which may be maintained in a separate repository by another team with its own review process. A legitimate exemption is usually paired with a narrower replacement rule scoped to the exempted area.

An enforcement level decides what a policy failure does to the run: advisory only reports it, the middle level stops the run but allows an authorized override, and the strictest level stops the run with no override. Sentinel and OPA use different words:

OPA has only two levels: soft-mandatory and hard-mandatory are Sentinel words, and mandatory is OPA's. An OPA policy with no enforcement_level is advisory, so a policy that clearly fails can still let the run apply with only a warning.

Two different permissions

• Manage policies — create, edit and delete policies and policy sets. It does not include the right to override.
• Manage policy overrides — continue runs after overridable failures in every workspace (it grants read access to workspaces for that purpose) without being able to edit policies. Give it to a change-advisory or governance team.

What an override does

Overriding is per run. On the run's page the user chooses Override and Continue; for a CLI-driven run with a soft-mandatory Sentinel failure, typing override at the apply prompt does the same. Nothing persists: the policy keeps its level, the workspace stays in scope, and the next run is evaluated again. An override never adds an exclusion or downgrades a policy.

When a hard-mandatory policy check fails

There is no override button, no CLI override and no permission that helps. The ways forward are to change the configuration so it complies, or to change the policy (or its level) in the policy set, and then queue a new run. If a run fails policies in several sets, each is judged separately and the non-overridable failure decides: overriding the OPA mandatory failure does not help while a Sentinel hard-mandatory check also fails.

HCP Terraform runs Sentinel in one of two modes: policy checks (the original mode) or policy evaluations (chosen by giving a Sentinel set the Agent policy set type). OPA sets always run as policy evaluations. The mode changes what a policy can read and how overrides work:

Two consequences are worth remembering. A budget rule built on tfrun.cost_estimate.delta_monthly_cost sees no cost data in an Agent-type set and must live in a policy-check set. And when a team needs both a pinned runtime and a cost rule, it splits them: the cost policy in a policy-check set, the rest in an Agent-type set with the pinned version. Even with the set-level option on, a failing run still stops at the policy stage first; the override is still per run.

A VCS-connected Sentinel policy set is a repository directory containing sentinel.hcl, which declares every policy and shared module the set uses. A .sentinel file that no policy block declares is ignored, even after HCP Terraform picks up the commit. VCS-connected sets update when the tracked branch changes; there is nothing to re-create or re-attach.

policy "restrict-regions" {
  source            = "./restrict-regions.sentinel"
  enforcement_level = "soft-mandatory"
}

module "helpers" {
  source = "./modules/helpers.sentinel"
}

• policy blocks — one per policy, with source and enforcement_level. Local policy files sit in the same directory as sentinel.hcl and are referenced by a relative path such as ./restrict-regions.sentinel. Remote sources must be HTTP or HTTPS URLs that return the raw policy text, such as the public registry's policy URLs; a URL that returns a web page displaying the file, rather than the file itself, fails, and non-HTTP source schemes are not accepted.
• module blocks — shared helper code (for example "find resources of a type, skip deletes"), local or HTTP(S). A policy uses it with import "helpers", so every policy that needs it shares one maintained copy. Declaring a helper as a policy would evaluate it instead of sharing it.
• Pre-written policies — HashiCorp and partners publish ready-made Sentinel policy libraries (for example CIS benchmark controls for the major clouds) in the public registry. Reference them by URL from a policy block at any enforcement level to get maintained standard checks without writing them.

Parameters: one policy, different values per set

A Sentinel policy file can declare param values with defaults (a param statement inside the .sentinel policy, not a block in sentinel.hcl). HCP Terraform supplies policy set parameters — key/value pairs stored on the policy set — to that set's Sentinel policies, and a supplied value replaces the default. Two project-scoped sets can therefore source the same policy file and give each team its own allowed list, with no hard-coded team lists in the code. Parameters keep secrets (such as an API token the policy needs) out of the repository: a parameter marked sensitive is write-once and is never displayed again. Parameters are for Sentinel only, are managed on the set (not on workspace variable pages), and hold values, not code.

OPA: policies.hcl, query and input

policy "no-public-ssh" {
  query             = "data.terraform.policies.no_public_ssh.deny"
  enforcement_level = "mandatory"
}

Each OPA policy names a Rego rule through query (package path plus rule name). The query must return an array: empty means pass, any element means fail, so a deny rule that collects messages is the usual pattern. Leave out enforcement_level and the policy is advisory. HCP Terraform passes OPA one input document: input.plan is the Terraform JSON plan (resource_changes, planned_values, …) and input.run is run metadata such as input.run.workspace (including auto_apply), input.run.project and input.run.organization. OPA sets do not receive policy set parameters.

Pick the data source that actually contains the fact the rule is about: configuration facts come from tfconfig/v2, planned values and actions from tfplan/v2, current real-world values from tfstate/v2, and run options and workspace metadata from tfrun.

Traps that make correct configurations fail

• References in tfconfig. A module that sets tags = var.tags has no constant_value for tags, only a reference, so a tfconfig rule looking for a literal owner key fails every resource the module builds. Value-based rules belong on change.after in tfplan/v2.
• Unknown values in planned_values. planned_values is an estimate of the resulting state and omits values not known until apply. An instance whose subnet_id references a subnet created in the same run looks as if the subnet is missing. Read resource_changes and treat an attribute marked in change.after_unknown as set.
• Deletes have no after. For a delete, change.after is null, so a negated lookup such as not r.change.after.tags["cost-center"] is true and the rule reports the resource being removed. resource_changes also lists no-op resources. Rules about desired values should first filter to actions that include "create" or "update".

Introduce a new rule at advisory level across its full scope, use the warnings to find and fix the affected workspaces, then raise the level. Advisory policies are evaluated on every run and their failures appear in run results, but they never stop a run — so the platform team gets an evidence list of who would be blocked without blocking anyone.

1. Advisory, at full scope. Trying the rule on one pilot workspace says nothing about how the rest of the estate would be affected.
2. Remediate. Owning teams fix their code within an agreed window.
3. Enforce with an exception path. Move to soft-mandatory (Sentinel policy check) or mandatory (OPA) and give a named change board Manage policy overrides, so new violations stop while approved exceptions can continue run by run.
4. Harden. Raise to hard-mandatory once no exceptions should remain.

Excluding the workspaces that are known to violate a rule removes enforcement exactly where it is needed; starting at hard-mandatory blocks teams before anyone knows the impact.

A risky infrastructure change is one that can destroy data or take down a shared dependency: a delete, a replacement, a destroy run, a partial (targeted) apply, or an unattended auto-apply in production. Policies identify them from the plan's actions and the run's metadata.

Change actions in the JSON plan

A rule that compares actions is ["delete"] misses both replacement orders. Test membership instead:

deletes = filter tfplan.resource_changes as _, rc {
  rc.type is "aws_db_instance" and
  rc.change.actions contains "delete"
}
main = rule { length(deletes) is 0 }

That catches removed blocks, destroy plans and every forced replacement, while letting in-place updates, ignore_changes edits and address changes declared with moved blocks through — a moved block re-addresses the existing object rather than deleting it, which is why it is the safe way to rename.

Run metadata

• tfrun.is_destroy — true when the run's plan is a destroy plan, whether queued from the UI, the API or terraform destroy. It does not catch ordinary runs that delete objects (removed blocks, forced replacement, -replace); pair it with a resource_changes rule.
• tfrun.target_addrs — the addresses of a targeted run (null when none). Requiring it to be null or empty forbids partial applies that can leave a stack half-updated.
• tfrun.replace_addrs, speculative, refresh, refresh_only — other run options; each answers a different question.
• In OPA, input.run.workspace.auto_apply together with input.run.project.name lets a mandatory deny rule refuse auto-apply in the production project.

No single control stops every way a critical resource can be lost, so production data stores get layers: workspace settings, plan-time policy, provider-side protection and a workspace layout that limits what any one run can touch.

A mandatory delete-blocking policy and provider deletion protection each cover all three accidents on their own; together they give defence in depth. ignore_changes never prevents deletion, and create_before_destroy still deletes the old object.

Workspace deletion

Deleting a workspace is different from destroying its resources. By default a workspace admin can delete only an unlocked workspace that manages no resources. An organization owner can force delete, which removes the workspace and its state but leaves the cloud resources running, now unmanaged. The safe order is a reviewed destroy run first, then deletion.

Limit the blast radius

A run can change only what is in its workspace's state. Splitting one large configuration into network, data and application workspaces (sharing values such as subnet IDs through outputs) means a routine application change cannot plan a network or database change at all, which needs far less review than catching such changes with policy. Moving code into modules does not help here: modules called from one workspace still share its state.