ISTQB CTFL Test Activities, Testware, Traceability and Test Roles

The test process is the set of test activities that testing needs in order to reach its objectives, tailored to the system and the project. The CTFL syllabus names seven groups of activities: test planning, test monitoring and control, test analysis, test design, test implementation, test execution and test completion. Without them, testing is less likely to achieve what it set out to do.

The activities are listed in a logical order, but they are not a waterfall. In practice they are often iterative (a team analyses, designs, implements and executes again in every sprint) or parallel (one feature is being executed while the next is still being analysed). Test monitoring and control is one of the activity groups, but it is not a step in the sequence: it runs throughout, checking every other activity.

Which activities a project includes, how they are carried out and when they happen is normally decided during test planning, based on the project's context (see the context section below). The activities exist in Agile and sequential lifecycles alike; in an iterative lifecycle the same groups simply recur in every iteration.

Test planning defines the objectives of testing and selects the approach that best meets them within the constraints of the context; monitoring and control then keep the work on course.

The distinction between monitoring and control is a frequent trap: monitoring measures, control acts. A progress report is evidence from monitoring; moving people or re-prioritising work in response to it is control. The two are usually named together because control decisions depend on monitoring data.

Planning is not a one-off. When monitoring shows the plan no longer fits, control actions can include updating the plan itself.

Test analysis decides what to test; test design decides how to test it. Both are often supported by test techniques, and both can find problems before a single test runs.

Two details are easy to misplace. First, coverage items belong to design, not analysis: analysis stops at test conditions, and design applies techniques to turn those conditions into coverage items and test cases. Second, analysis can produce defect reports: if a tester finds an acceptance criterion so vague that no expected result can be derived from it, that is a defect in the test basis, and it is reported so the business representative can correct it. Analysing the test basis is an early chance to find defects before any code is run.

Test implementation gets everything ready to run, test execution runs it and records what happened, and test completion wraps up and preserves what is worth keeping.

Test implementation

• Creates or acquires the testware needed for execution, such as test data.
• Organises test cases into test procedures and often assembles them into test suites; creates manual and automated test scripts.
• Prioritises and arranges the test procedures in a test execution schedule.
• Builds the test environment, including stubs, drivers, simulators and service virtualisation, and verifies it is set up correctly.

Design says what data and environment are required; implementation creates them.

Test execution

• Runs the tests according to the execution schedule (test runs), manually or automated, in forms that include continuous testing and pair testing sessions.
• Compares actual results with expected results and logs the results.
• Analyses anomalies to identify their likely causes, so they can be reported based on the failures observed.

Test completion

Test completion usually happens at project milestones such as a release, the end of an iteration or the completion of a test level. Its tasks:

• Create change requests or product backlog items for any unresolved defects.
• Identify testware that may be useful in the future and archive it or hand it over to the appropriate teams.
• Shut the test environment down to an agreed state.
• Analyse the test activities to identify lessons learned and improvements for future iterations, releases or projects.
• Create a test completion report and communicate it to stakeholders.

Test completion is backward-looking: it closes out work that has finished and captures what the next iteration or project can reuse and learn from.

Testing is never done in isolation: the context it runs in decides how the test process is tailored. Testing is funded by stakeholders and has to serve their business needs, so the best way to test depends on the situation.

Classifying a factor matters: team skills are a team members factor, architecture is a technical factor and regulations belong to the business domain, even though all of them constrain the project in a loose sense. The syllabus's project constraints are scope, time, budget and resources.

These factors affect the test strategy, the test techniques used, the degree of test automation, the required level of coverage, the level of detail of test documentation and reporting.

Factors often pull in different directions, and a good tailoring satisfies each one where it applies rather than letting one win everywhere. A safety-critical domain argues for rigorous, formal testing of the safety functions; a team with little testing experience argues for more guidance and simpler, well-explained techniques. Applying both where they matter serves the project better than picking just one.

Testware is the set of work products created by the test activities, and each activity has its own characteristic outputs. Organisations name and organise testware differently, and proper configuration management keeps it consistent and intact (configuration management itself is covered in the test management chapter).

A risk register is a list of risks with their likelihood, impact and mitigation information. It is a planning output. A test charter (used in exploratory testing) is a design output, alongside test cases. The useful chain to remember runs: conditions (analysis) → cases and charters (design) → procedures, suites and scripts (implementation) → logs and defect reports (execution) → completion report (completion).

Note that defect reports appear twice: analysis reports defects found in the test basis, execution reports defects revealed by failures.

Traceability is the maintained link between the elements of the test basis, the testware for those elements, the test results and the defects found, and it is what makes test monitoring and control effective.

Which direction you trace decides which question you can answer:

Coverage tells you what has tests; it cannot tell you how much risk remains, because a covered requirement may still have failing or unrun tests. Residual risk needs the results linked to the risks.

Good traceability also:

• Makes it possible to determine the impact of changes: when a requirement changes, you can find the test cases, results and defects linked to it.
• Facilitates test audits and helps meet IT governance criteria.
• Makes test progress and completion reports easier to understand, because they can include the status of test basis elements (requirements tested, passed, failed, pending).
• Helps communicate technical aspects of testing to stakeholders and supports assessment of product quality, process capability and progress against business goals.

CTFL describes two principal roles: the test management role, which is responsible for the test process as a whole, and the testing role, which is responsible for the technical work of testing.

Roles are not job titles. Who holds a role is decided by the tasks the person performs:

• How the test management role is carried out varies with the context. In Agile development, some test management tasks may be handled by the Agile team itself; tasks that span several teams or the whole organisation may be done by test managers outside the development team.
• Different people can take the roles at different times. The test management role can be performed by a team leader, a development manager and others; a business analyst who writes and runs acceptance tests is performing the testing role.
• One person can take on both the testing role and the test management role at the same time.

In other words, the roles describe tasks, not job titles: who carries them out depends on the context.