ISTQB CTFL Configuration Management in Testing: Configuration Items, Baselines and Change Control

Configuration management (CM) in testing is the discipline of identifying, controlling and tracking work products as configuration items. It answers a simple but essential question at any moment in a project: exactly which versions of which things were used, and what has changed since?

Without CM, a team cannot say which build a test ran against, which version of a test script produced a result, or which test data set a test case loaded. With it, every result can be tied to precise versions, approved items are protected from uncontrolled change, and earlier testing can be repeated.

CM is one of several management disciplines in the test management chapter, and each has a different object. Keeping them apart is a large part of understanding this topic:

A useful test: if the problem is about which version of a work product was used, or about work products changing without anyone controlling the change, it is a configuration management problem. If it is about what went wrong in the software, how much could go wrong, or how far testing has got, it belongs to one of the other disciplines.

In testing, both the testware and the test items are configuration items. The syllabus lists the work products concerned as test plans, test strategies, test conditions, test cases, test scripts, test results, test logs and test reports; all of these are testware. Test items, the individual parts of the test object being tested, are configuration items too.

Two consequences follow:

• CM is not only for the software under test. Results are produced by testware running against test items, so both sides must be identified and versioned. Controlling only one side leaves results that cannot be tied back to everything that produced them.
• Management documents count as testware. Test plans, test strategies and test reports are configuration items just as much as test cases and scripts. Test data and the test environment are also typically managed this way.

The ISTQB glossary defines a configuration item as an aggregation of work products that is designated for configuration management and treated as a single entity in the CM process. A configuration item can therefore be a single test script, a whole test suite or an entire test environment. Related terms differ:

For a complex configuration item, such as a test environment, CM records the items it consists of, their relationships and their versions. Take a test environment for a mobile banking app: a device emulator image, a backend API build, a message queue, a mock of the card provider and a test automation framework. Recording it as a configuration means writing down which of those components it contains, how they connect to each other, and the exact version of each.

That record is what makes the environment reproducible. Suppose the team needs to rebuild the environment months later to compare a new release against an old one, or a second team must set up an identical copy in another location. With the configuration recorded, both can assemble exactly the same components at exactly the same versions. Without it, they can only approximate the original and hope the differences do not matter.

The configuration record describes what the environment is made of. What happened while tests ran in it is recorded elsewhere, in the testware such as test logs.

A baseline is a formally approved version of a configuration item. In the syllabus's words, once a configuration item is approved for testing, it becomes a baseline, and from then on it can only be changed through a formal change control process.

Approval is the defining feature. Consider a test strategy document that goes through drafts 0.1 to 0.4 while the team discusses it. Drafting order does not matter: none of the drafts is a baseline until the stakeholders formally approve one, say draft 0.4 as version 1.0. From that moment version 1.0 is the baseline, and whatever is newer or older than it is just another version.

Formal change control governs every later change to a baseline. The order is what matters: the change is requested, assessed, approved and recorded before it is made, and making it produces a new, identified version of the item. A change that is made first and documented afterwards has not been controlled, however carefully it is written up, and the same applies to removing part of a baseline and adding it back later.

Change control is not a freeze. Baselines change all the time on real projects, whenever a corrected or improved version is needed; the process simply ensures each change is deliberate, approved and traceable.

CM keeps a record of the changed configuration items when a new baseline is created so that the team can revert to a previous baseline and reproduce previous test results. This is the payoff of the whole discipline.

Typical situations where it matters:

• Isolating the cause of a changed result. A security test passed in the spring baseline and fails in the summer one. The record of what changed between the two baselines narrows the search to those items, and reverting to the spring baseline confirms that the old result can still be obtained.
• Demonstrating that a reported result was genuine. A supplier questions a failure reported against its component. Reverting to the baseline that was in force when the failure was reported lets the tester show the failure again on the same versions.
• Comparing releases fairly. To compare response times between two releases, both must be measured under the configuration that applied to each, not under whatever happens to be installed today.

Reproducing a result means recreating the same conditions: the same versions of the test object, of the test environment's components, and of the testware and test data used. Documents that describe testing, such as plans and reports, say what was done but cannot by themselves recreate it; only the versioned items in the baseline can.

Because the purpose is to return to earlier states, superseded baselines are kept, not deleted. Each approved change produces a new baseline and the earlier one stays available. The change record supports change control; it never replaces it.

To properly support testing, the syllabus says CM ensures two things.

1. Configuration items are identified, versioned, tracked and related

All configuration items, including test items, are:

• uniquely identified,
• version controlled,
• tracked for changes, and
• related to other configuration items, so that traceability can be maintained throughout the test process.

Version control and change tracking let the team know which version of an item produced which result and retrieve any earlier version. Relating items is a separate guarantee: each item's own history says nothing about which other items it was used with. For example, a test report that must state which test case versions were run on which environment configuration, or a team that must find every automated script written against an old interface version of a component, needs links between configuration items, not just the history of each one.

2. Items are referenced unambiguously in testware

All identified documentation and software items are referenced unambiguously in testware. A test charter that says "use the staging configuration file", or a log entry that says a run used "the newer browser driver", fails this rule when several versions exist: different people can legitimately pick different ones. Every reference should name the item's unique identifier and version.

Each guarantee answers a different question, which is how the exam tells them apart:

In DevOps, automated configuration management is normally included in the pipeline. Continuous integration, continuous delivery, continuous deployment and their associated testing are typically implemented as an automated DevOps pipeline (described in the lesson on testing in DevOps and the SDLC), and CM runs inside it.

Frequent builds make CM more important, not less. Each pipeline run combines a particular version of the code, the build scripts, the environment definition and the automated tests. When the pipeline itself identifies and records those versions on every run, any build can later be traced to exactly what it contained and any earlier result can be reproduced, at a pace no manual process could match. Infrastructure defined as versioned code is a typical way pipelines make even the test environment a recorded configuration item.

Configuration management is not a single step; it supports every test activity by keeping its work products identified and versioned:

CM supplies the identification, versioning and relationships that these activities rely on; it does not itself decide what to test, how much, or in what order.