ISTQB CTFL Experience-based Test Techniques: Error Guessing, Exploratory and Checklist-based Testing

Experience-based test techniques derive tests from the knowledge, skills and experience of the tester rather than from a specification (black-box) or the code's structure (white-box). The CTFL syllabus names three of them:

• Error guessing — anticipating errors, defects and failures from what the tester knows, applied methodically as a fault attack.
• Exploratory testing — designing, executing and evaluating tests at the same time while learning about the test object, sometimes conducted as session-based testing.
• Checklist-based testing — designing and running tests to cover the test conditions listed in a checklist.

All three find defects that formal techniques can miss, and all three depend on who is doing the testing: the same technique in less experienced hands finds less. Their general role alongside black-box and white-box techniques is covered in the Test Techniques Overview lesson; this lesson explains each technique in detail and how to tell them apart.

Error guessing is a technique that anticipates the occurrence of errors, defects and failures based on the tester's knowledge. The tester asks "what is likely to go wrong here?" and designs tests aimed at exactly those weak spots — even when the specification says nothing about them. Submitting the same order twice in quick succession, because similar web shops have been known to charge customers twice, is a typical example.

The syllabus lists three sources of that knowledge:

• How the application has worked in the past — earlier releases and their history.
• The types of errors developers tend to make, and the types of defects that result from those errors.
• The types of failures that have occurred in other, similar applications.

What makes a test error guessing is its source: the tester's knowledge of likely mistakes and failures, used to aim tests at the places most likely to break.

Anticipated errors, defects and failures are commonly grouped into six categories. Classifying an anticipated error means asking where the mistake lives:

Some errors could be argued into two categories; pick the one that describes the error most directly. A missing case in the decision logic is logic, not computation; a mismatch in what one component passes to another is an interface problem, not an input problem.

A fault attack is a methodical approach to error guessing. Instead of guessing ad hoc, the tester works in two steps:

1. Create or acquire a list of possible errors, defects and failures. The list comes from experience, from defect and failure data (for example, failures seen in earlier releases or in competitors' products) and from common knowledge about why software fails.
2. Design tests that will identify the defects, expose them or cause the failures on the list.

The list is what defines a fault attack: every entry names a defect or failure to target, and every test is designed against one of those entries.

Exploratory testing is testing in which tests are simultaneously designed, executed and evaluated while the tester learns about the test object. Each result shapes the next test: the tester notices something odd, designs a probe, runs it, and uses what she learns to decide where to go next. Nothing is fully scripted in advance.

The syllabus gives exploratory testing three purposes:

• Learning more about the test object.
• Exploring it more deeply with focused tests.
• Creating tests for untested areas.

Because tests are designed in the moment, its value lies in discovery and learning; pre-defined, repeatable checks remain the job of scripted test cases.

Session-based testing is a structured way to run exploratory testing. Its elements are:

• Time-box — the session runs for a defined period agreed in advance (for example 90 minutes).
• Test charter — states the objectives of the session and guides the tester. The objectives can be treated as high-level test conditions.
• Coverage items — identified and exercised during the session.
• Test session sheet — the tester may use session sheets to document the steps followed and the discoveries made as the session runs.
• Debriefing — after the session, a discussion between the tester and the stakeholders interested in the results.

A charter and a time-box give structure and accountability. The tests themselves are still designed in the moment, so each session follows the path the tester's learning leads to.

Exploratory testing is most useful when specifications are few or inadequate or when there is significant time pressure on testing. It is also useful to complement other, more formal test techniques — it runs alongside them rather than replacing them.

It is more effective when the tester is experienced, has domain knowledge and brings essential skills such as analytical skills, curiosity and creativeness. The results depend on the tester as much as on the charter.

Exploratory testing can incorporate other test techniques. Probing the smallest and largest file sizes an upload accepts during a session is boundary value analysis used inside exploration; trying inputs with emoji and accented characters because similar text fields have failed on them before is error guessing used inside exploration. The activity remains exploratory testing.

In checklist-based testing a tester designs, implements and executes tests to cover the test conditions found in a checklist. Checklists are built from experience, from knowledge about what is important for the user, and from an understanding of why and how software fails.

Checklist items are often phrased as questions, and each item should be checkable separately and directly. Items may refer to requirements, graphical interface properties, quality characteristics or other forms of test conditions. A good item is specific: "Does the date picker allow keyboard-only selection of a date?"

Three kinds of item do not belong in a checklist:

• Items that can be checked automatically ("Does every link on the page resolve?" — a link checker can do this).
• Items better suited as entry or exit criteria ("Have all critical defects been closed before release?").
• Items that are too general ("Is the screen user-friendly?").

Checklists support functional and non-functional testing alike — usability heuristics and accessibility guidelines are common checklist bases.

A checklist wears out. As developers learn to avoid the errors it targets, its entries become less effective at revealing defects. Keep it useful by:

• Adding new entries for newly found high-severity defects, written as specific, directly checkable items.
• Updating it regularly based on defect analysis.
• Not letting it grow too long, so it stays focused on what matters.

Regular updates are what keep a checklist finding defects release after release.

When there are no detailed test cases, checklists still provide guidelines and some consistency. The level of detail is a trade-off:

Specific items suit a team that needs comparable results from different testers; high-level items suit a team that values breadth over comparability.

Each technique has telltale elements. Identify them from what the tester works from and what the tests aim at:

The near-twin to watch is the fault attack versus the checklist: a fault-attack list holds defects and failures to target; a checklist holds test conditions to cover.

The techniques combine well. A checklist gives consistent coverage of known conditions, exploratory sessions find what nobody listed, and error guessing points both toward likely weak spots. Defects found in exploration can feed new checklist items or fault-attack lists.