ISTQB CTFL Risk Management: Risk Level, Project and Product Risks, Risk-based Testing
Risk management in the ISTQB CTFL syllabus is the pair of activities, risk analysis and risk control, that organizations use, supported by testing, to find, rate and respond to risks so that test effort goes where failure would hurt most. This lesson covers risk likelihood, impact and risk level (quantitative and qualitative), how to tell project risks from product risks and the four categories of project risk, how product risk analysis shapes test scope and thoroughness, the four responses to product risks, mitigation by testing, and risk monitoring.
On this page9 sections
- What is risk management in software testing?
- Risk likelihood, risk impact and risk level
- Project risks versus product risks
- The four categories of project risk
- Consequences of product risks
- Product risk analysis: identification and assessment
- How product risk analysis shapes test scope and thoroughness
- Product risk control: mitigation, acceptance, transfer, contingency
- Risk monitoring
- Identify a risk's level from its likelihood and impact, using the quantitative (likelihood × impact) or qualitative (risk matrix) approach
- Distinguish project risks from product risks, and place project risks in the organizational, people, technical or supplier category
- Recognize the consequences of project risks and of product risks when they occur
- Describe risk analysis (identification and assessment) and risk control (mitigation and monitoring), and define risk-based testing
- Explain how product risk analysis influences test scope, levels, types, techniques, coverage, effort and priority
- Explain the responses to analyzed product risks (mitigation by testing, acceptance, transfer, contingency plan) and the testing actions that mitigate them
What is risk management in software testing?
Risk management is the set of activities an organization uses to deal with uncertain events that could stop it achieving its objectives. Done well, it increases the likelihood of achieving those objectives, improves the quality of the product and increases stakeholders' confidence and trust.
The CTFL syllabus divides risk management into two main activities, each with two parts:
| Main activity | Made up of | What it produces |
|---|---|---|
| Risk analysis | Risk identification + risk assessment | A list of risks, each categorized, rated and prioritized, with proposed ways to handle it |
| Risk control | Risk mitigation + risk monitoring | Actions that reduce risk levels, and evidence that those actions work |
A test approach in which test activities are selected, prioritized and managed on the basis of risk analysis and risk control is called risk-based testing. Most of what follows explains how that works for product risks.
Risk likelihood, risk impact and risk level
A risk is a potential event, hazard, threat or situation whose occurrence causes an adverse effect. Every risk is characterized by exactly two attributes:
- Risk likelihood: the probability that the risk occurs. It is greater than zero and less than one. Something with a probability of zero is not a risk, and something certain to happen (probability one) is a fact or an issue to handle, not a risk.
- Risk impact (harm): the consequences if it does occur.
Together these two attributes express the risk level, which is the measure of the risk. The higher the risk level, the more important it is to treat the risk.
Calculating risk level
In the quantitative approach, risk level = risk likelihood × risk impact. With both rated on a 1–5 scale, a risk with likelihood 3 and impact 5 has a risk level of 15; one with likelihood 2 and impact 4 has a level of 8. You multiply; you do not add or average the ratings.
Ranking by risk level can give a different order from ranking by impact or likelihood alone. For example:
| Risk | Likelihood | Impact | Risk level (L × I) |
|---|---|---|---|
| P | 4 | 4 | 16 |
| Q | 3 | 5 | 15 |
| S | 5 | 3 | 15 |
| T | 2 | 3 | 6 |
Risk P has the highest level and is treated first, even though Q has a higher impact and S a higher likelihood.
Qualitative and mixed approaches
In the qualitative approach, likelihood and impact are rated on ordinal scales such as low, medium and high, and a risk matrix combines each pair into a risk level (for example, high likelihood with high impact gives a high level). It still produces a risk level for every risk; what makes it qualitative is that no number is calculated. Organizations can also mix the two approaches.
Project risks versus product risks
Project risks relate to the management and control of the project; product risks relate to the quality characteristics of the product being built. The test for classification is what is threatened: the project's ability to deliver (schedule, budget, scope), or the product's behaviour and quality.
| Project risk | Product risk | |
|---|---|---|
| Relates to | Managing and controlling the project | Quality characteristics of the product (for example those in ISO/IEC 25010) |
| Typical examples | Delays in delivering work products, staff leaving, wrong estimates, budget cuts, a supplier failing to deliver | Missing or wrong functionality, incorrect calculations, runtime errors, poor architecture, inefficient algorithms, inadequate response time, poor user experience, security vulnerabilities |
| If it occurs, it affects | The project's schedule, budget or scope | Users, the business and third parties (see consequences below) |
| Usually handled by | Management action (re-planning, staffing, contracts) | Testing and other quality measures, or acceptance, transfer or a contingency plan |
Be careful with risks that sound technical. “The build pipeline may be unavailable during the final test cycle” is a project risk: it threatens how the project runs, not how the product behaves. “The currency converter may apply yesterday's exchange rate” is a product risk. Compatibility, maintainability and portability are quality characteristics too, so risks such as “the app may fail on an older operating system version” are product risks.
One chain of events can involve both kinds. If the only experienced code reviewer leaves and reviews stop, that departure is a project risk occurring; if an unreviewed security flaw then ships, the flaw is the product risk occurring. Who ends up affected does not change the category of the cause.
The four categories of project risk
The CTFL syllabus groups project risks into four categories:
| Category | Examples |
|---|---|
| Organizational issues | Delays in delivering work products, inaccurate estimates, cost-cutting |
| People issues | Insufficient skills, conflicts, communication problems, shortage of staff |
| Technical issues | Scope creep, poor tool support |
| Supplier issues | A third party failing to deliver, bankruptcy of a supporting company |
Two placements surprise people. Scope creep (requirements keep being added, enlarging what must be built and tested) is listed under technical issues, not organizational ones. And bankruptcy of a supporting company is a supplier issue: the category follows the cause (a third party), not what that party provided.
When a project risk occurs it affects the project's schedule, budget or scope, and so the project's ability to achieve its objectives. It does not directly change the product's quality characteristics, although it can reduce the testing that would have found defects.
Consequences of product risks
When a product risk occurs, the product fails in a way that harms the people and organizations that rely on it. The syllabus lists these possible consequences:
- User dissatisfaction
- Loss of revenue, trust or reputation
- Damage to third parties
- High maintenance costs and overload of the help desk
- Criminal penalties
- In extreme cases, physical damage, injuries or even death
Contrast these with project-risk consequences, which are about the project itself: a slipped release date, overspent test budget or reduced test scope. Customers abandoning an app after repeated crashes is a product-risk consequence; the test budget running out before the last test cycle is a project-risk consequence.
Product risk analysis: identification and assessment
The goal of product risk analysis, from a testing perspective, is to provide awareness of product risk so that test effort is focused in a way that minimizes the residual level of product risk. Some residual risk always remains. It should ideally begin early in the SDLC, so its results can shape testing from the start, and it is revisited as more is learned rather than fixed once. It applies to every kind of system, not only safety-critical ones.
Product risk analysis consists of two activities:
| Activity | What happens | Typical techniques |
|---|---|---|
| Risk identification | Generating a comprehensive list of risks | Brainstorming, workshops, interviews, cause-effect diagrams |
| Risk assessment | Categorizing the identified risks, determining their likelihood, impact and level, prioritizing them, and proposing ways to handle them | Quantitative calculation, qualitative risk matrix, or a mix |
So rating a risk's likelihood and impact, or proposing how to handle it, is assessment, not identification. A stakeholder workshop listing what could go wrong is identification.
Why categorize? Categorization helps assign mitigation actions, because risks in the same category can usually be mitigated with a similar approach (for example, several security risks may all be addressed by security testing by a specialist). Categories do not set the test order; risk level and priority do.
How product risk analysis shapes test scope and thoroughness
Product risk analysis influences both how much is tested and how thoroughly. Its results are used to:
- Determine the test scope to be carried out
- Determine the particular test levels and propose the test types to perform
- Determine the test techniques to employ and the coverage to achieve
- Estimate the test effort required for each task
- Prioritize testing in an attempt to find critical defects as early as possible
- Determine whether any activities in addition to testing could be employed to reduce risk
In practice, a higher-risk area gets more rigorous techniques, higher coverage targets and earlier testing; a lower-risk area gets lighter treatment. The intensity of testing follows the risk level, area by area.
Product risk control: mitigation, acceptance, transfer, contingency
Product risk control comprises all measures taken in response to identified and assessed product risks. It consists of risk mitigation (implementing the actions proposed during risk assessment to reduce the risk level) and risk monitoring. The syllabus names four response options:
| Response | What it means | Example |
|---|---|---|
| Risk mitigation by testing | Reduce the risk level through testing activities | More rigorous testing of a high-risk calculation module |
| Risk acceptance | Record the risk and consciously take no further action | A low-likelihood, low-impact cosmetic risk |
| Risk transfer | Move the consequences to another party | Buying insurance that covers losses from a data breach |
| Contingency plan | Prepare what to do if the risk occurs | A fallback hosting supplier ready to take over if the main one fails |
A single risk can receive more than one response. Acceptance is a deliberate decision, not neglect; the risk stays on the register.
Mitigating product risks by testing
The syllabus lists these actions:
- Select testers with the right level of experience and skills for the risk type (for example a security specialist for security risks)
- Apply an appropriate level of independence of testing
- Conduct reviews and perform static analysis
- Apply appropriate test techniques and coverage levels
- Apply appropriate test types that address the affected quality characteristics
- Perform dynamic testing, including regression testing
Matching the test type to the quality characteristic matters. A risk that an app fails on older operating system versions is a compatibility risk, so it calls for compatibility testing across the supported versions; more functional tests on a single device would leave it untouched. Each mitigation action works on the product or the work products that describe it.
Testing mitigates product risks. Project risks such as a budget cut or a supplier delivering late need management action (re-planning, staffing, escalation); extra dynamic testing does nothing for them.
Risk monitoring
Risk monitoring is the part of risk control that keeps the risk picture current once mitigation is under way. Its aims are to:
- Ensure the mitigation actions are effective (for example, checking whether new test results still justify a mitigated risk's rating)
- Obtain further information to improve the risk assessment
- Identify emerging risks, such as those introduced by a third-party integration added mid-project, and record them
Do not confuse it with the other activities:
| Activity | Focus |
|---|---|
| Risk identification | Producing the initial comprehensive list of risks |
| Risk assessment | Rating likelihood, impact and level; prioritizing; proposing responses |
| Risk mitigation | Carrying out the actions that reduce the risk level |
| Risk monitoring | Checking those actions work, refining the assessment, catching emerging risks |
| Test monitoring | Checking test progress against the test plan (a sibling topic) |
Spotting a new risk during monitoring feeds back into analysis, so risk management is a continuous loop rather than a one-off step in test planning.
Tip. Questions on this topic are mostly short recall and understanding items, often with an uppercase qualifier such as BEST, MOST, FIRST or CORRECT. Expect to name the two attributes of a risk and what likelihood means, calculate a risk level from likelihood and impact ratings and rank several risks by it, and recognize a qualitative risk-matrix approach. Classification items give risks from a risk register and ask which are project or product risks (sometimes Select TWO), which project risk category a situation falls into, and whether a consequence comes from a project or a product risk. Other items ask which activity a description matches (identification, assessment, mitigation or monitoring, as distinct from test monitoring), why risks are categorized, the goal and timing of product risk analysis, how its results influence testing (sometimes as a roman-numeral combination item), which response a decision represents, and which testing action or test type mitigates a given product risk.
- Risk management = risk analysis (identification + assessment) and risk control (mitigation + monitoring); risk-based testing selects, prioritizes and manages test activities on that basis.
- A risk has two attributes: likelihood (probability greater than zero and less than one) and impact (the harm). Together they give the risk level.
- Quantitative risk level = likelihood × impact; qualitative uses a risk matrix; the two can be mixed. The higher the level, the more important the treatment.
- Project risks concern managing and controlling the project and hit schedule, budget or scope; product risks concern quality characteristics and hit users, revenue, reputation, third parties and safety.
- Project risk categories: organizational (delays, inaccurate estimates, cost-cutting), people (skills, conflicts, communication, staff shortage), technical (scope creep, poor tool support), supplier (delivery failure, bankruptcy).
- Product risk analysis aims to minimize residual product risk by focusing test effort, and should start early in the SDLC.
- Its results drive test scope, levels, types, techniques, coverage, effort estimates, prioritization, and whether non-testing activities could also reduce risk.
- Responses to product risks: mitigation by testing, acceptance, transfer and contingency plan; they can be combined.
- Mitigation by testing: suitably skilled testers, appropriate independence, reviews and static analysis, suitable techniques and coverage, test types for the affected quality characteristic, dynamic and regression testing.
- Risk monitoring checks mitigation is effective, improves the assessment and identifies emerging risks.
Frequently asked questions
How is risk level calculated in ISTQB CTFL?
In the quantitative approach, CTFL calculates risk level as risk likelihood multiplied by risk impact. A risk rated 3 for likelihood and 5 for impact on 1 to 5 scales has a risk level of 15. In the qualitative approach, likelihood and impact are rated on scales such as low, medium and high and combined with a risk matrix instead of a calculation.
What is the difference between a project risk and a product risk?
A project risk relates to the management and control of the project, such as staff shortages, late deliveries or a failing supplier, and affects schedule, budget or scope. A product risk relates to the product's quality characteristics, such as incorrect calculations, slow response times or security vulnerabilities, and affects users, revenue, reputation or safety.
What are the four categories of project risk in CTFL?
The CTFL syllabus lists organizational issues (delays, inaccurate estimates, cost-cutting), people issues (insufficient skills, conflicts, communication problems, staff shortage), technical issues (scope creep, poor tool support) and supplier issues (third-party delivery failure, bankruptcy of a supporting company).
What is risk-based testing?
Risk-based testing is a test approach in which test activities are selected, prioritized and managed on the basis of risk analysis and risk control. Higher-risk areas get earlier, more rigorous and more thorough testing so that the residual level of product risk is minimized.
What are the responses to product risks in CTFL?
The CTFL syllabus lists four responses to analyzed product risks: risk mitigation by testing, risk acceptance (recording the risk and taking no further action), risk transfer (moving the consequences to another party, for example by contract) and a contingency plan (preparing what to do if the risk occurs). Responses can be combined.
What is the difference between risk assessment and risk monitoring?
Risk assessment is part of risk analysis: it categorizes identified risks, rates their likelihood, impact and level, prioritizes them and proposes how to handle them. Risk monitoring is part of risk control: it checks that mitigation actions are effective, gathers information to improve the assessment and identifies emerging risks.
Can testing reduce project risks?
Testing mitigates product risks, not project risks. Project risks such as a budget cut, inaccurate estimates or a supplier going bankrupt are handled by management actions like re-planning, staffing changes or escalation, although project risks can still reduce how much testing gets done.
Source
This lesson covers the "Managing the Test Activities" domain of the official CTFL exam guide. Vendors revise their guides — check the source for the current version.
Sign up free to mark lessons complete, bookmark topics and track your exam readiness.