ISTQB CTFL Test Techniques Overview: Black-box vs White-box vs Experience-based

A test technique is a systematic procedure for deciding what to test and how to test it. Techniques support the tester in two activities of the test process: test analysis, which answers what to test by identifying test conditions, and test design, which answers how to test by elaborating those test conditions into test cases.

The reason techniques exist is that exhaustive testing is impossible: no team can try every combination of inputs and preconditions. A technique gives a systematic way to derive a relatively small but sufficient set of test cases, so effort goes into the tests that matter instead of random or duplicated ones. The point is method, not guarantees: a technique narrows the input space in a repeatable way, while deciding which tests run first remains a test management decision.

During analysis and design, techniques help the tester define three things:

• Test conditions — testable aspects of the test object (what to test).
• Coverage items — the attributes the technique says must be exercised, such as equivalence partitions, boundary values, decision table columns, transitions, statements or branches. Coverage is measured against them.
• Test data — the concrete input values needed to exercise those items.

Other work products of the test process belong to planning, monitoring and completion activities rather than to applying a technique.

The CTFL syllabus classifies test techniques into exactly three categories: black-box, white-box and experience-based. Each category is defined by the source the tests are derived from.

The aliases are paired in this order: black-box = specification-based, white-box = structure-based. A common slip is to reverse them, or to treat "experience-based" as another name for one of the first two — it is its own category.

A black-box technique is one based on an analysis of the specified behavior of the test object without reference to its internal structure. The tester treats the object as a box whose inside is not looked at: tests come from what the object is supposed to do, as described in requirements, specifications, user stories or acceptance criteria.

Two consequences follow, and both are tested:

• Black-box test cases are independent of the implementation. If the implementation changes but the required behavior stays the same — a rewrite, a refactoring, a replaced algorithm — black-box test cases are still useful without change.
• Black-box tests can be designed early. They need only the specification, so they can be written as soon as requirements are approved, before any design or code exists.

The flip side: when the required behavior changes — for example a new business rule is added — black-box tests must be added or revised for it, because their basis has changed. Black-box techniques also have defined coverage items taken from the specification (partitions, boundary values, decision table columns, transitions), so coverage can be measured. The individual techniques are taught in the black-box techniques lesson.

A white-box technique is one based on an analysis of the test object's internal structure and processing. The tester looks inside the box: tests are derived from the code, the detailed design and its control flow — its statements, decisions and branches.

Because the test cases depend on how the software is designed, two consequences follow:

• White-box test cases can only be created after the design or implementation exists. With approved requirements but no design or code, there are no branches or statements to analyse, so white-box tests cannot be written yet.
• White-box test cases change when the implementation changes. If code is restructured or an algorithm replaced, tests derived from the old branches and statements need revision even though the behavior is the same.

White-box techniques also have their own coverage items (statements, branches), which only exist in the code. Statement and branch testing, and the value of white-box testing, are taught in the white-box techniques lesson.

An experience-based technique is one that uses the knowledge and experience of testers to design and implement test cases — knowledge of the application, the business domain, how similar systems have failed and which mistakes developers typically make. Predicting defects from mistakes the tester has met before is the classic example (error guessing); exploratory testing and checklist-based testing are the other two experience-based techniques in the syllabus.

The syllabus makes two statements about this category:

• Effectiveness depends heavily on the tester's skills. Black-box and white-box techniques give a systematic procedure, so two testers working from the same specification or code tend to derive similar test cases. With experience-based techniques, results vary with the person: what one tester finds depends on what they know and have seen.
• They are complementary to black-box and white-box techniques. They can detect defects that the systematic techniques may miss. Full coverage of partitions or branches shows that those coverage items were exercised, not that no defects remain; a tester's knowledge of likely mistakes can still find more.

Complementary means "in addition to", not "instead of": experience-based testing does not replace the other categories, because the systematic techniques in turn find defects an exploratory session may miss. Their coverage is less formally defined and harder to measure than for the systematic techniques, although checklist-based testing can still give some consistency and a degree of coverage. The individual techniques are taught in the experience-based techniques lesson.

The categories differ in their basis, when tests can be created, how they react to change and how much they depend on the individual tester. This comparison is the core of the learning objective.

As a general rule, judge structural and behavioral changes separately: a structural change affects tests derived from the structure, a behavioral change affects tests derived from the specification.

Classify a technique by asking what its tests are derived from: the specification, the code, or the tester's knowledge.

• Black-box: equivalence partitioning, boundary value analysis, decision table testing, state transition testing.
• White-box: statement testing, branch testing.
• Experience-based: error guessing, exploratory testing, checklist-based testing.

Checklist-based testing is a frequent trap: a checklist looks systematic, but its items come from experience of what matters to users and why software fails, so it is experience-based. Decision table testing is another: "decision" refers to business rules in the specification, not to decisions in the code, so it is black-box — whereas branch testing uses the code's decisions and is white-box.

The same logic classifies a described activity. Designing tests from a requirements document without opening the code is black-box. Analysing a module's control flow is white-box. Relying on what the tester knows about how software typically fails is experience-based.

Collaboration-based test approaches are not a fourth category of test technique. The syllabus covers them in a separate section and calls them approaches: they focus on avoiding defects through collaboration and communication among testers, developers and business representatives — writing user stories collaboratively, agreeing acceptance criteria, and acceptance test-driven development (ATDD).

ATDD and agreeing acceptance criteria with the business are therefore collaboration-based approaches. Inside such an approach the team may still apply black-box or experience-based techniques to the acceptance criteria — the approach describes how the team works together, while the categories describe what tests are derived from. These approaches are taught in the collaboration-based test approaches lesson.

Other terms that sound like categories but are not: static and dynamic testing (and reviews) describe kinds of testing, functional and non-functional are test types, risk-based testing is a test management approach, requirements-based names a test basis, and exploratory testing is one individual technique inside the experience-based category, not a category itself.

Choose a category by checking what is actually available and what the tests must achieve:

• The test basis is a specification → black-box techniques apply, and they work before design or code exists. Their tests also survive changes to the implementation.
• The basis is the code or detailed design → white-box techniques apply, once that structure exists.
• The basis is people's knowledge → experience-based techniques; their value rises when documentation is weak and falls with less experienced testers.
• Measurable coverage is required → prefer a category with defined coverage items (black-box or white-box), and note which of those items exist yet.
• Systematic tests are done → experience-based testing still adds value, because coverage achieved is not the same as defects absent.

In practice the categories are combined over a project: black-box tests from the requirements first, white-box tests once the code exists, and experience-based testing alongside both. No single category is sufficient on its own.