Terraform Professional: Sharing Data Between Projects and Managing Provider and Module Versions

There are four practical ways for one Terraform project to use values another project produces, and they differ mainly in what the consumer must be allowed to read and how tightly it is coupled to the producer's state layout:

Choose by the constraint that matters most. Any approach that reads state requires state access, so it is out when consumers may not see the producer's state. An approach that names a backend, state key or workspace is coupled to that location, so prefer a published key or a direct lookup when the producer's layout is expected to change. A published store is only useful if consumers can already read it with the credentials they hold. Inside HCP Terraform with no other constraint, tfe_outputs is HashiCorp's recommendation.

Configuring the backend itself and the HCP Terraform remote state sharing setting are covered in the remote backends lesson; this lesson is about choosing and using the data path.

terraform output reads root module output values from the current state and prints them; its format flag decides whether a script can use the result directly.

-raw only supports strings, numbers and booleans. For a list, use -json and let jq unpack it:

terraform output -json private_subnets | jq -r '.[]'

# all outputs saved once, then read later
terraform output -json > outputs.json
jq -r '.app_url.value' outputs.json

The shape difference catches people out: with a name, -json prints the bare value; without a name, every value sits under .<name>.value. The argument is an output name, not an expression; select elements from the JSON instead.

Only outputs recorded in state can be read. A new output added to configuration appears after an apply (a normal apply, or one in refresh-only mode) writes it; a plan alone shows "Changes to Outputs" but does not record them.

Marking an output sensitive = true only hides it in Terraform's human-readable display; the value is still stored in plain text in state and is printed in plain text whenever it is asked for explicitly.

Two consequences follow. Anyone who can run terraform output against a state can read its secrets, so CI read access to state is secret access. And a pipeline that publishes outputs as a build artifact must filter on the flag itself, because -json includes everything:

# keep only entries whose sensitive flag is false
terraform output -json > all.json
jq 'with_entries(select(.value.sensitive | not))' all.json

Backend encryption at rest protects the stored object from people without access; it does nothing against an authorised reader. The rules for how sensitivity propagates through expressions, and ephemeral values that never reach state, are covered in the configuration-design lessons.

terraform_remote_state exposes only the producer's root module outputs to the reading configuration, but to get them it must fetch the entire state snapshot, so the reader's credentials need access to everything in that state.

data "terraform_remote_state" "net" {
  backend = "s3"
  config = {
    bucket = "acme-tf-state"
    key    = "network/terraform.tfstate"
    region = "eu-west-1"
  }
}

# only root outputs are visible
subnet_ids = data.terraform_remote_state.net.outputs.private_subnet_ids

Four facts decide most questions about it:

• Whole-object access. A state file is one object. There is no way to grant a reader just the outputs section in a bucket or blob policy, so each reader can see every resource attribute, including secrets in resources the producer never meant to share.
• Copies spread. Whatever a data source returns is saved in the consumer's state. A password read through remote state now lives in plain text in every consumer's state too.
• Root outputs only. A child module's output (module.vpc.private_subnet_ids) is invisible to readers until the producer declares a root output that passes it through and applies. There is no outputs.vpc.… path into child modules.
• Latest applied snapshot. Readers see what the producer's last apply recorded, not what its configuration currently says.

# producer root module: pass the child value through, then apply
output "private_subnet_ids" {
  value = module.vpc.private_subnet_ids
}

Reading remote state does not lock it; it is a read of the latest snapshot. Because of the access problem, the terraform_remote_state documentation recommends publishing data explicitly instead, or using tfe_outputs in HCP Terraform.

There are two ways to share values without giving consumers any state access: the producer publishes the values to a store, or the consumer looks up the real objects with provider data sources.

Publish the values

Publishing data means the producer writes the specific values consumers need to a purpose-built store, and each consumer reads them with that store's data source, so access is granted per value rather than per state file.

# producer
resource "aws_ssm_parameter" "vpc_id" {
  name  = "/shared/network/prod/vpc_id"
  type  = "String"
  value = aws_vpc.main.id
}

# consumer
data "aws_ssm_parameter" "vpc_id" {
  name = "/shared/network/prod/vpc_id"
}

The terraform_remote_state documentation lists stores that work this way: Consul key/value, Kubernetes ConfigMaps, DNS records, cloud parameter stores and object storage. Pick the one the consumers can already reach with the credentials they hold. A service project that has only Kubernetes cluster credentials can read a kubernetes_config_map but not an SSM parameter; a project with AWS credentials but no state access can read SSM parameters.

The published keys become an interface: the producer can refactor its configuration, split into several projects or change backends without breaking consumers, as long as it keeps writing the same keys.

Look up the infrastructure directly

A consumer can skip the producer's Terraform entirely and read the real objects from the provider API with data sources, which removes every dependency on the producer's state, backend, workspace and project layout.

data "aws_vpc" "prod" {
  tags = {
    Environment = "prod"
    Role        = "shared"
  }
}

This is the most resilient choice when the producer is about to be split, renamed or migrated to another backend, and when consumers must not see any state. The cost is that the lookup criteria become the contract. Single-object data sources such as aws_vpc fail the plan when the filters match more than one object, and fail when they match none. Teams must keep identifying tags unique (or filter on more attributes) and treat changes to those tags as breaking changes.

In HCP Terraform, tfe_outputs moves the data between workspaces and run triggers move the timing; they are independent, and a design usually needs both.

tfe_outputs

The tfe_outputs data source (in the hashicorp/tfe provider) reads a workspace's root outputs through the HCP Terraform API without full state access. Its two attributes behave differently:

data "tfe_outputs" "db" {
  organization = "acme"
  workspace    = "db-prod"
}

endpoint = data.tfe_outputs.db.nonsensitive_values.db_endpoint # visible
password = data.tfe_outputs.db.values.db_password              # stays redacted

A root output built from values must itself be marked sensitive or the plan fails. Wrapping the whole values map in nonsensitive() would unmark the real secrets too, and a sensitive output is simply absent from nonsensitive_values.

Run triggers

A run trigger connects a downstream workspace to one or more source workspaces. When a run in a source workspace applies successfully, HCP Terraform queues a run in the downstream workspace. Commits, plans, and discarded or failed runs in the source do not trigger anything, because upstream infrastructure has not changed.

• The trigger is configured on the downstream workspace, which names its sources.
• Triggered runs wait for confirmation unless the downstream workspace enables Auto-apply run triggers. That setting is separate from the workspace's general auto-apply, so you can auto-apply runs caused by upstream changes while runs from VCS commits still need approval.
• Run triggers share no data. The downstream configuration still reads the new values itself, typically with tfe_outputs.

The external data source (from the hashicorp/external provider) runs a local program and returns its result as a map of strings; HashiCorp describes it as an escape hatch for read-only lookups that no provider supports.

data "external" "rack" {
  program = ["python3", "${path.module}/rack.py"]
  query   = { host = var.host }
}
# data.external.rack.result["rack"]

The program must follow a strict protocol:

• It reads the query as a JSON object on stdin.
• It writes one JSON object whose values are all strings to stdout and exits 0. Numbers, lists and nested objects are rejected, so encode them as strings (for example a comma-separated list) and convert back with functions such as split or tonumber.
• On failure it writes a message to stderr and exits non-zero; Terraform shows that message.
• It must have no observable side effects. Terraform re-runs it every time it reads the data source, which happens on every plan, including speculative plans.

Anything that creates something — a ticket, a record, a credential — belongs in a managed resource of a provider, which records the object in state and creates it once. Nothing in the data source's configuration changes how often the program runs.

A version constraint is a comma-separated list of conditions; Terraform selects the newest available version that satisfies every condition from every module that constrains the same dependency.

Pre-releases such as 6.0.0-beta2 never match >, >=, <, <= or ~>; you must name the exact version.

Constraints combine. The root module and every child module may declare required_providers versions for the same provider, and Terraform treats them equally. If no single release satisfies all of them, terraform init fails; one configuration installs exactly one version of each provider, so a module cannot get its own. Work an example with the google provider: root ~> 6.10, module A >= 6.8, < 6.14, module B != 6.12.0, releases 6.9.1, 6.11.3, 6.12.0, 6.13.2, 6.14.0 → 6.9.1 fails the root, 6.12.0 fails module B, 6.14.0 fails module A, leaving 6.11.3 and 6.13.2; with no lock file Terraform installs the newest, 6.13.2. A conflict with a module you do not own is fixed by moving to a module release whose constraint allows your provider version.

Where each style belongs, following HashiCorp's guidance:

• Reusable modules declare only the minimum provider version they need (>= 5.30). An upper bound or pin in a shared module narrows every caller's choice.
• Root modules use ~> for providers so they get both a floor and a ceiling, and let the lock file pin the exact release.
• Third-party modules are pinned to an exact version so upgrades arrive only when you change the code.

The Terraform CLI's own required_version and staged upgrades of Terraform itself are covered in the version-upgrade lesson.

.terraform.lock.hcl records the exact provider versions and package checksums terraform init selected, so every machine that uses the committed file installs the same, verified provider builds. It does not track modules.

Selection behaviour

So when a constraint changes, run terraform init -upgrade where the change is made and commit the new lock file with it. Running -upgrade in CI, or deleting the lock file there, means CI installs versions nobody reviewed.

Checksums and platforms

Each entry carries hashes. zh: hashes are the registry's signed checksums of the official .zip packages, one per platform; h1: hashes are computed from an installed package.

• A provider first installed from its origin registry gets zh: hashes for every platform the provider is published for, plus an h1: for the local package — so a lock file made on macOS usually verifies on Linux CI.
• A provider first installed through a filesystem or network mirror gets only the installing platform's h1: hash, because a mirror offers no signed checksum list. Another platform then fails verification ("doesn't match any of the checksums previously recorded").
• Fix that without weakening verification by pre-recording the other platforms, then commit:

terraform providers lock \
  -platform=linux_arm64 \
  -platform=windows_amd64

Changing the install method later does not remove hashes already recorded. A plugin cache only reuses packages; it does not add checksums.

How you pin a module depends on its source: the version argument works only for registry sources, and Git sources pin with ?ref=.

module "dns" {
  source = "git::https://code.internal.example/platform/dns-zone.git?ref=9b2e71c04d5a3f86e1b7c29d0a4f5e3b8c6d1a72"
}

Branches and tags are movable names: someone with repository rights can point them at different code. A full commit SHA identifies exactly one commit, so the module can change only when the calling configuration changes — the strongest pin when tags cannot be trusted. Only ref selects the revision; other URL parts control where in the repository Terraform looks and how it clones.

For third-party registry modules, an exact version turns each upgrade into a reviewed pull request; relying on ~> plus the lock file does not, because the lock file never holds module versions. Publishing modules to a private registry is covered in the module and registry lessons.