ISTQB CTFL White-box Test Techniques: Statement Coverage, Branch Coverage and the Value of White-box Testing

White-box test techniques derive test cases from the internal structure of the test object — its code, its control flow and its processing — rather than from its specified behaviour. They are also called structure-based techniques. The CTFL syllabus covers the two most popular and simplest code-related white-box techniques: statement testing and branch testing.

Both techniques reason about the code as a control flow graph: nodes stand for statements (or blocks of straight-line statements) and the arrows between them are the possible transfers of control, which show the sequences in which the statements can execute. Each technique picks a different element of that graph as its coverage item:

In each case the aim is the same: design test cases that exercise the coverage items until an acceptable level of coverage is reached. The syllabus also notes that more rigorous white-box techniques exist for safety-critical, mission-critical and high-integrity systems, and that white-box ideas are used beyond code (for example API testing, or neuron coverage in neural network testing). Those are outside the Foundation Level scope; how white-box techniques differ from black-box and experience-based ones in general is covered in the Test Techniques Overview lesson.

Statement coverage is the number of executable statements exercised by the tests divided by the total number of executable statements in the test object, expressed as a percentage. Only executable statements are coverage items, in the count and in the total.

Coverage is measured over the whole test suite, and each statement counts once, however many tests run it. When a new test is added, only the statements it exercises for the first time raise the figure. For example, if a component has 120 executable statements, the existing suite exercises 84 of them (70%) and a new test exercises 30 statements of which 18 were already covered, the suite now exercises 84 + 12 = 96 distinct statements: 96 ÷ 120 = 80%. Adding the raw counts (84 + 30) double-counts the overlap, and quoting the new test on its own ignores the rest of the suite.

100% statement coverage guarantees one thing: every executable statement in the code has been executed at least once. So any statement that contains a defect has been run, and running it may cause a failure that shows the defect. No defect can hide simply because its statement never executed.

It does not guarantee two things the exam likes to probe:

• Data-dependent defects can survive. Executing a statement does not detect every defect in it. A statement such as cents = INT(amount * 100) may give the right result for every positive amount and round the wrong way only for negative amounts (refunds). Tests that run it with 4.25 and 9.10 cover the statement fully but never trigger the failure. Coverage says the statement ran with some data, not with the data that exposes the defect.
• Decision logic can be left untested. A decision outcome that contains no statements of its own — typically the false outcome of an IF with no ELSE — can be skipped without lowering statement coverage.

READ retries
IF retries > 3 THEN
  locked = TRUE
  CALL alert
ENDIF

One test with retries = 5 runs every executable statement: 100% statement coverage. The false outcome (retries ≤ 3) was never taken, so nothing tells the tester that the code never clears locked on that route — an account locked by an earlier call stays locked. A defect that shows only when a decision goes the other way is invisible to statement coverage. Exercising every outcome is what branch testing adds.

A branch is a transfer of control between two nodes in the control flow graph. It is not a statement (a node), not a condition (the expression evaluated at a decision) and not a path (a whole route from entry to exit). The syllabus names two kinds, and both are coverage items:

• Unconditional branches — straight-line flow from one node to the next, taken whenever the first node is executed.
• Conditional branches — the outcomes of a decision. They typically correspond to:
  • the true and the false outcome of an IF…THEN (whether or not there is an ELSE);
  • each outcome of a CASE/switch statement, including the default;
  • the decision to continue or to exit a loop.

Unconditional branches are coverage items too. Different tools and authors draw control flow graphs slightly differently, so Foundation-level items give you branch counts as numbers rather than asking you to count arrows in a diagram.

Branch coverage is the number of branches exercised by the tests divided by the total number of branches, expressed as a percentage. 100% branch coverage means every branch, conditional and unconditional, has been exercised at least once — so every outcome of every decision has been taken: true and false for each IF, every case plus the default for each CASE, and both continuing and exiting for each loop.

The counting rules are the same as for statements: count across the whole suite, and count each branch once. Suppose a control flow graph has 30 branches, 6 of them unconditional. Test A exercises 18 branches; test B exercises 14, of which 9 are also exercised by A. Together they exercise 18 + (14 − 9) = 23 distinct branches, so branch coverage is 23 ÷ 30 ≈ 77%. Count the overlap only once.

Full branch coverage of a decision does not require testing every combination of conditions inside it (for example both parts of a > 0 AND b > 0 in every true/false pairing). It only needs the decision as a whole to come out true at least once and false at least once. Testing condition combinations belongs to more rigorous techniques outside the Foundation syllabus.

Branch coverage subsumes statement coverage: any set of tests that achieves 100% branch coverage also achieves 100% statement coverage, but not the other way round. Every executable statement lies on at least one branch, so exercising every branch necessarily runs every statement. A decision outcome with no statements, however, can be skipped while every statement still runs.

A useful consequence for reading coverage reports: a component can genuinely show 100% statement coverage with less than 100% branch coverage, but a report of 100% branch coverage with less than 100% statement coverage is impossible — it signals a measurement or reporting error. Likewise, a test set can never need more tests for statement coverage than for branch coverage.

Branch coverage counts each branch once, regardless of which branches are combined on the same run. Some defects only fail on a specific path — a particular sequence or combination of decision outcomes — and branch coverage never demands any specific combination. Consider:

READ p, q
IF p > 0 THEN
  idx = 10
ELSE
  idx = 1
ENDIF
IF q > 0 THEN
  idx = idx - 1
ELSE
  idx = idx + 1
ENDIF
value = table[idx]   // table holds entries 1 to 10

Two tests, (p = 5, q = 3) and (p = −5, q = −1), take all four decision outcomes and look up entries 9 and 2, so statement and branch coverage are both 100% and both tests pass. The out-of-range lookup (entry 11) happens only on the path where p > 0 is true and q > 0 is false — a combination neither test follows. Covering every path would catch it, but the number of paths grows multiplicatively with each decision and loops make it practically unbounded, which is why path coverage is not the Foundation-level goal.

Branch coverage also inherits the data limitation of statement coverage: a branch can be taken with values that happen not to trigger a data-dependent defect.

Trace the pseudocode and ask, for each target, how many separate executions are needed. One execution follows exactly one route, so it takes exactly one outcome of each decision it reaches (a loop is the exception: its decision is evaluated repeatedly within one run).

1. Statement coverage: list the statements that sit on mutually exclusive outcomes (THEN vs ELSE, different CASE values). You need at least as many tests as the largest set of statements that can never run together; statements that are always reached ride along.
2. Branch coverage: list every decision outcome, including outcomes with no statements (a missing ELSE, a CASE default). Each test takes one outcome per decision it reaches, so for independent decisions in sequence the minimum is the largest number of outcomes of any single decision. Nested decisions need more, because the inner decision is reached only through one outer outcome.
3. Check: the branch minimum is never smaller than the statement minimum.

A frequent trap: for independent sequential decisions, the number of paths (outcomes multiplied together) is not the branch minimum — that confuses branch coverage with path coverage.

The fundamental strength of white-box techniques is that the entire software implementation is taken into account during testing. Because tests come from the code itself, they can find defects even when the specification is vague, outdated or incomplete — for example, when a specification says nothing about how errors are handled, reaching full branch coverage still forces tests through every error-handling branch the developer wrote.

The corresponding weakness is the mirror image: if the software does not implement one or more requirements, white-box testing may not detect the resulting defects of omission. Missing logic has no statements or branches, so 100% coverage can be reached without any test noticing the gap. Finding it needs tests derived from the requirements — a black-box strength.

Two further points complete the picture:

• Static use. White-box techniques are not limited to executing code. They can be used in static testing, for example in dry runs of code. They suit code that is not yet ready for execution, as well as pseudocode and other high-level or top-down logic that can be modelled with a control flow graph. White-box techniques need the implementation, or a model of its control flow — not a specification of required behaviour.
• Objective coverage measurement. Performing only black-box testing provides no measure of actual code coverage. White-box coverage measures give an objective measurement of how much of the code was exercised, show which code was not exercised, and supply the information needed to generate additional tests that increase coverage and, in turn, confidence in the code.