Terraform Professional (002): Registry Modules and terraform test

A registry module is called by a short registry source address, not a URL. The public Terraform Registry uses three parts, NAMESPACE/NAME/PROVIDER; a private registry (HCP Terraform or Terraform Enterprise) puts the registry hostname in front, HOSTNAME/ORGANIZATION/NAME/PROVIDER. The order is fixed, and swapping any two parts makes Terraform look for a different module that usually does not exist.

Two common mistakes:

• Using the repository name. A module published from the repository terraform-azurerm-network is addressed as …/network/azurerm: the registry splits the repository name into its NAME and PROVIDER parts. The repository name itself is never a registry address.
• Getting the order wrong. In the private form the organization comes straight after the hostname, then the module name, then the provider.

Nested modules with a double slash

Many registry packages ship nested modules under modules/, each with its own README. To call one directly, append // and the path to it inside the package:

module "consul_cluster" {
  source  = "hashicorp/consul/aws//modules/consul-cluster"
  version = "0.12.0"
}

The double slash separates the package (which is versioned and downloaded as a whole) from the subdirectory within it. With a single slash Terraform reads the whole string as one malformed registry address. A ?ref= query does not select a subdirectory — it is the Git source syntax for choosing a revision.

The version argument on a module block is accepted only when source is a registry address (public or private). Git, GitHub, HTTP, S3/GCS and local-path sources have no version metadata for Terraform to resolve, so a version argument beside them makes terraform init fail. Pin those sources another way — for Git, a ?ref= tag or commit in the source string. Terraform never compares a Git ref against a version constraint.

Modules are installed by terraform init into .terraform/modules. Any change to a module block's source or version means different module code is needed, so plan and validate stop with an error saying the installed module no longer matches the configuration until you run terraform init again. Other commands do not help:

• terraform validate and terraform fmt work on what is already installed and never download modules.
• terraform providers lock and the .terraform.lock.hcl file cover providers only; there are no module entries to add or delete.
• terraform init -upgrade lets Terraform select newer versions within the existing constraints. It cannot move past an exact pin, and it cannot make version valid for a non-registry source.

Version constraint operators and pinning strategy are covered in the lesson on providers and versioning; this lesson is about the registry side of the workflow.

Constant variables in module source and version (Terraform 1.15)

From Terraform 1.15, source and version on a module block can use input variables declared const = true and local values built from them, instead of only literal strings. This lets one root module choose a module release per environment. Modules are installed during terraform init, before any plan, so the value must be a known constant at that point — that is exactly what const declares.

variable "net_version" {
  type  = string
  const = true
}

locals {
  net_release = var.net_version
}

module "network" {
  source  = "app.terraform.io/acme/network/aws"
  version = local.net_release
}

Other variable flags do not substitute: ephemeral = true keeps a value out of state and plan files, and nullable = false rejects null. Neither makes a variable available during module installation.

Terraform CLI downloads private registry modules with an API token for the registry's hostname, taken from the CLI configuration or the environment. Consumers do not need access to the module's source repository, and the module block has no token argument.

• Credentials block in the CLI configuration file (~/.terraformrc, or the credentials.tfrc.json file that terraform login writes):

  credentials "tfe.example.com" {
    token = "<user or team API token>"
  }

• TF_TOKEN_<hostname> environment variable, with each period in the hostname written as an underscore — TF_TOKEN_app_terraform_io. A hyphen may be written as a double underscore when the shell or CI system rejects hyphens: tfe-eu.corp.example becomes TF_TOKEN_tfe__eu_corp_example. An environment variable takes precedence over a credentials block for the same host.
• terraform login <hostname> opens a browser and prompts for a token. It suits a laptop; a non-interactive CI job should use the environment variable instead.

Use a user or team API token. Organization tokens are for managing teams and workspaces and cannot be used for CLI actions such as module downloads; for CI the usual choice is a team token stored as a pipeline secret. Some variables look similar but serve other tools: TFE_TOKEN is read by the tfe provider, and a provider "tfe" block configures resource management, not module installation.

The registry documents every provider and module per released version, and the page opens on the latest release by default. When you build or update a configuration, first switch the documentation to the version your configuration actually uses — the provider version recorded in the lock file, or the module version you pinned.

Copying an example or input from a newer release than you run is a frequent cause of terraform validate reporting an unsupported argument. Fixes that do not work:

• terraform init -upgrade stays inside the constraint, so a pin to 5.x or to an exact module version cannot reach an argument that only exists in a newer major version than the constraint allows.
• Deleting the lock file and re-initializing selects the newest version that still satisfies the constraint — the same major version.
• lifecycle { ignore_changes } only affects planned updates; an argument the schema does not define still fails validation.
• Declaring a variable of the same name in the root module adds nothing to the child module's interface.

A module's registry page lists its inputs (required ones have no default; optional ones show their default), outputs, provider requirements, resources, nested modules and examples. The smallest working module block sets source, version (for registry modules) and the required inputs only; every optional input falls back to its default. A description does not make an input required.

The public registry publishes modules straight from GitHub. A repository qualifies only when all of these hold:

• It is on GitHub and public.
• Its name is terraform-<PROVIDER>-<NAME>, for example terraform-aws-vpc or terraform-google-network. The registry derives the module's NAME and PROVIDER from this.
• It has a description and follows the standard module structure: the root module's .tf files in the repository root, with a README, and any nested modules under modules/ and examples under examples/.
• It has at least one release tag that is a semantic version, x.y.z, optionally prefixed with v.

After publishing, the registry adds a new module version for each new valid tag and ignores other tags:

Publishing to an HCP Terraform private registry

HCP Terraform and Terraform Enterprise publish private modules from a connected VCS repository, by one of two methods. Publishing needs organization owner rights or the Manage private registry permission, plus admin access to the repository so the registry can set up its webhook.

Several modules in one repository

A monorepo with modules/web, modules/cache and modules/dns is published as three registry modules. For each, set the Source Directory (where the module lives) and, for tag-based publishing, a Module Tag Prefix such as cache-. A tag cache-v3.0.4 then publishes version 3.0.4 of the cache module only. Without prefixes, a plain v3.0.4 tag is a release of every module published from the repository. Tags are repository-wide, and the prefix is the only part of a tag the registry uses to tell modules apart — anything after the version number is read as part of the version itself.

Removing a bad version

Deleting a Git tag does not remove the version it published: the registry keeps serving it, and constraints like ~> 1.4 still resolve to it. Delete the version on the module's page in the private registry. Changing the publishing method or removing the webhook leaves already published versions in place.

terraform test is Terraform's built-in framework for validating a module: it plans or applies the configuration under test with given inputs and checks the results with assertions. Tests are written in .tftest.hcl (or .tftest.json) files, which Terraform discovers in the module's root directory and in the tests directory. Test files in the root directory are always loaded.

-target is a plan option for resource addresses and does not select test files, and terraform -chdir=… changes which directory is the module under test.

Run blocks

A test file contains run blocks, executed in order, plus optional variables, provider, mock_provider, override blocks and a test block for file-wide settings. A test file has no module or data blocks of its own. Each run block sets:

• command — apply by default, or plan. An apply run creates real infrastructure with the credentials available; Terraform then attempts to destroy everything the file created once the file finishes. Use command = plan for checks that must create nothing.
• plan_options — plan mode, refresh, replace and target for the run.
• variables, providers, module, assert, expect_failures, state_key and parallel, described below.

Variables in tests and their precedence

Test runs take input variables from several places. Values set in the test file win over everything outside it, and a run's own block wins over the file's:

1. variables block inside a run block
2. File-level variables block in the test file
3. Values from outside the file: -var and -var-file options, automatic .tfvars files and TF_VAR_ environment variables (in their usual order)

So a -var option in CI does not override a value written in the test file; a run without its own value inherits the file-level one.

Automatic variable files have a directory scope. A terraform.tfvars in the module's root directory applies to every test. A terraform.tfvars in the test directory applies only to test files in that directory, where it overrides the root file; test files in the root directory still use the root values.

An assert block holds a condition and an error_message. Conditions can reference the module's resources, data sources, variables and outputs, and earlier runs' outputs as run.<run name>.<output>.

In a plan run a condition can only use values that are known at plan time. Attributes set from configuration (an S3 bucket's bucket argument) are known; computed attributes such as id or arn are unknown until apply, and an assertion on them fails with an unknown condition value. Assert on the configured attribute instead, or use an apply run (or a mocked value generated during plan) when the computed value really matters. try() cannot turn an unknown into a known value, and refresh = false only affects reading objects that already exist.

run "archive_named" {
  command = plan

  assert {
    condition     = aws_s3_bucket.archive.bucket == var.archive_name
    error_message = "Archive bucket does not use the requested name."
  }
}

Setup and helper modules

A run can execute an alternate module instead of the configuration under test, with a nested module { source = "./tests/setup" } block. Alternate modules must be local paths or registry modules. Use them to create prerequisites (a bucket, a VPC) or to load and verify results, and pass their outputs into later runs:

run "setup" {
  module {
    source = "./tests/setup"
  }
}

run "create" {
  variables {
    bucket = run.setup.bucket_name
  }
}

Inside a test file the reference is always run.setup…, never module.setup….

State and cleanup

Terraform keeps one state file per module, not per test file: the main configuration has one, and each alternate module has its own, shared by the runs that use that module. When the file finishes, Terraform destroys resources in reverse run-block order, so objects created by later runs are removed before the setup resources they depend on — a bucket that cannot be deleted while it holds objects is destroyed last. Every resource the file created, including those from alternate modules, is destroyed.

Testing custom conditions with expect_failures

expect_failures lists the checkable objects whose custom conditions must fail; the run then passes when they fail and fails when they do not. It is how you prove a validation rule rejects bad input.

Name the object that carries the condition: a resource precondition that tests two variables is listed as the resource, because the variables have no validation of their own.

Two limits matter:

• Only custom conditions count. A type conversion error (a string for a number variable) or a provider error is not a custom condition, so it still fails the run.
• Failures are expected only in the run's own command. Variable validation and most preconditions fail during planning, so test them with command = plan. In an apply run, a failure during the plan step is unexpected and the run fails. Assertions are never reached when planning stops, so an assert cannot prove a validation failure.

A mock_provider block (Terraform 1.7+) replaces a provider with one that never calls the real API: it accepts the same configuration and schema, and generates values for computed attributes. Unit tests then run fast, without credentials, and can use command = apply without creating anything real.

• Values set in configuration are kept; only computed attributes are generated.
• With no defaults, computed strings get a random 8-character alphanumeric value, numbers get 0 and booleans false — they do not follow the provider's real formats (no vpc- prefix, no valid ARN).
• mock_resource "<type>" and mock_data "<type>" inside the mock provider set defaults for a managed resource type or a data source type respectively. Mock definitions can also be kept in .tfmock.hcl files referenced with source.
• override_resource, override_data and override_module (at file level or inside a run) replace the values of one specific resource, data source or module call, mocked provider or not.

When mock values exist: override_during

Mocked and overridden values are generated during apply by default, so in a plan run they are still unknown and an assertion on them fails. Set override_during = plan on the mock or override block (or on the mock provider) to generate them during the plan.

Mixing real and mocked providers

One test file can hold fast mocked runs and a real integration run. Because a provider "aws" block and a mock_provider "aws" share the same provider name, give one of them an alias, then choose per run with the providers attribute:

provider "aws" {}

mock_provider "aws" {
  alias = "fake"
}

run "unit" {
  providers = {
    aws = aws.fake
  }
}

run "integration" {}

The module under test is not edited: provider selection belongs to the run block.

Run blocks execute one after another by default. Setting parallel = true on run blocks (or once in the file's test block, which applies it to every run) lets eligible runs execute together. Two runs can only run in parallel when all of these hold:

• both are marked parallel = true;
• neither references the other's outputs (a run.b.vpc_id reference creates a dependency);
• they do not share a state file.

The command mode and run-level variables play no part. The state-file rule is the one that catches teams out: two runs that test the main configuration share its state unless they set different state_key values. A run using an alternate module already has its own state.

state_key works the other way too. Giving the same key to runs that use different modules makes them share one state file — for example, when one module's run should see and act on the objects another module's run already created. Output references alone never share state: they pass values, while each run still plans against its own state file.

To limit concurrency without editing files, use terraform test -run-parallelism=n. -parallelism=n is a different setting: it limits concurrent operations inside each run's plan or apply.

HCP Terraform can run a private registry module's terraform test suite for you. Test integration is available only for branch-based modules: once enabled, HCP Terraform runs the tests on every commit to the designated branch, and publishing stays a separate, manual step where someone selects a tested commit as a new version. It is not available for tag-based modules or with the single-repository publishing option.

Credentials for registry tests

Tests that create real resources need provider credentials, which come from the module's own test settings, not from the workspaces that consume the module:

• Test environment variables set on the module (for example AWS keys), marked sensitive where needed.
• Dynamic provider credentials configured for the module's tests, which exchange a workload identity token for short-lived cloud credentials. Subject claims can be scoped to the specific module, so no long-lived keys are stored and access is limited to those test runs.

Note what does not supply provider credentials: CLI credentials blocks and TF_TOKEN_ variables authenticate Terraform to HCP Terraform, not a provider to a cloud, and a mock provider removes the need for credentials only by no longer creating real infrastructure.

Remote runs from the CLI

terraform test -cloud-run=app.terraform.io/globex/storage/google runs the local test files remotely in HCP Terraform and associates the run with that module. The address must be a private registry module, and its hostname must match the registry that hosts it — not registry.terraform.io. The CLI must be logged in to that host, and -junit-xml cannot be combined with -cloud-run; the results are viewed in HCP Terraform. A -cloud-run is an on-demand extra run; it does not replace the per-commit tests.