CTFL sample questions — 25 answered and explained

Verification and validation are defined by the reference used (specification or stakeholder needs), not by static or dynamic testing.

Verification compares with specified requirements; validation compares with users' and other stakeholders' needs. Both can use static or dynamic testing. A stakeholder review of a requirements specification asks whether it reflects their needs, so it is validation without any execution. That makes ii and iv false, and leaves i and iii.

1. Classify each statement by its reference: specification or stakeholder needs.
2. Recall that static and dynamic testing can each support both verification and validation.
3. Mark ii and iv false because they tie validation to execution.
4. Select the combination of i and iii.

Exam tip: Verification vs validation is about the yardstick, not whether the software runs.

Typical test objectives include verifying compliance with contractual, legal and regulatory requirements.

Among the typical test objectives are verifying that specified requirements are fulfilled and verifying that the test object complies with contractual, legal and regulatory requirements. Removing defects is debugging, assigning fixes is management, and proving a defect-free product is impossible.

1. Recall the list of typical test objectives.
2. Reject activities that belong to debugging or project management.
3. Reject the impossible goal of proving no defects remain.

Exam tip: Checking contractual, legal and regulatory compliance is a typical test objective.

One typical test objective is providing information to stakeholders to allow them to make informed decisions.

Several objectives are related, but the product owner is asking for information to decide whether to release. That is the objective of informing stakeholders' decisions. Building confidence and reducing risk are outcomes of testing, and coverage measures thoroughness; none of them is what the request is for.

1. Identify what the stakeholder wants: information for a release decision.
2. Match it to the objective about informed decisions.
3. Separate it from the related objectives of confidence, coverage and risk reduction.

Exam tip: Test results exist to let stakeholders make informed decisions, such as whether to release.

Typical test objectives cover finding defects, evaluating work products and validating the test object, not correcting the code.

Three options are on the list of typical test objectives. Diagnosing a root cause and correcting the code is the debugging process, which testing triggers but does not include.

1. Recall the typical test objectives.
2. Spot the option that describes changing the code.
3. Classify that option as debugging.

Exam tip: Testing finds defects; correcting them is debugging, not a test objective.

Typical test objectives include verifying regulatory compliance and reducing the risk of inadequate quality.

Compliance with legislation and reducing the risk of wrong payments are both on the list of typical test objectives. Locating and correcting the cause of a defect are debugging steps, and proving that no defects remain is something testing cannot achieve.

1. Recall the typical test objectives.
2. Remove the goals that are debugging steps: locating and correcting defects.
3. Remove the goal of proving the absence of defects.
4. Select the compliance and risk-reduction goals.

Exam tip: Compliance and risk reduction are test objectives; locating and fixing defects is debugging.

Good practices for all SDLCs: a test activity for every development activity; different objectives per test level; test analysis and design starting in the corresponding development phase; reviews of drafts as early as possible.

Pairing every development activity with a test activity means everything the team produces is subject to quality control. Each distractor is a reversed version of another good practice: test levels should have different objectives, analysis should start during the matching development phase, and reviews should start on drafts.

1. Recall the four good practices that apply to every SDLC.
2. Check each option against them.
3. Notice that three options invert a good practice.
4. Choose the corresponding-test-activity practice.

Exam tip: Every development activity gets a matching test activity.

Different test levels have specific and different test objectives, which allows testing to be appropriately comprehensive while avoiding redundancy.

If every level had the same objectives, the same things would be tested repeatedly while other risks were missed. Giving each level its own objectives spreads the coverage and avoids duplicated effort. The distractors describe staffing, scheduling and reuse decisions that do not follow from having distinct objectives.

1. Ask what problem distinct objectives solve.
2. Recognise the twin goals: comprehensive coverage and no redundancy.
3. Reject options about staffing, scheduling or reuse.

Exam tip: Different objectives per level: thorough coverage without duplication.

Testers are involved in reviewing work products as soon as drafts of this documentation are available, so earlier testing and defect detection can support shift left.

The earlier a defect in a specification is found, the cheaper it is to fix. Reviewing drafts lets testers raise ambiguities and gaps before they are built into the code. Each distractor waits for a later event, a baseline, the start of a test level or a build, and so loses that early-detection benefit.

1. Recall the principle of early testing.
2. Ask which option allows the earliest review.
3. Choose reviewing as soon as drafts are available.

Exam tip: Review drafts, don't wait for the final version.

Test-driven development (TDD) directs coding through test cases instead of extensive software design: tests are written first, then the code is written to satisfy them, and then tests and code are refactored.

The developer-level loop of test, code, refactor is the defining pattern of TDD. BDD and ATDD are also test-first, but they start from business behaviour or acceptance criteria and usually work at a higher level than a single small test. Feature-driven development is a development method rather than a test-first approach.

1. Notice that the test comes before the code.
2. Notice the test is a small developer test, not an acceptance criterion or a scenario.
3. Choose test-driven development.

Exam tip: TDD: test, code, refactor.

Defects that never cause a failure during execution, or that sit in work products that cannot be executed, can only be found by static testing.

Dynamic testing finds defects through failures, so unreachable code and a design pattern that was implemented differently but still works produce no failure to observe. Dynamic testing can also only be applied to executable work products, so an ambiguity in a requirements document is out of its reach. The memory shortage and the boundary result are both visible failures when the software runs, so dynamic testing can find them.

1. Mark items that sit in non-executable work products: iv.
2. Mark code items that cannot produce a failure when run: i and ii.
3. Recognise iii and v as failures that execution reveals.
4. Choose i, ii and iv.

Exam tip: No failure to observe, or nothing to execute, means only static testing can find the defect.

Because static testing can be performed early, it creates a shared understanding among the stakeholders involved and improves communication between them.

When business representatives, developers and testers examine the same requirements together, misunderstandings surface and are resolved while they are still cheap. The syllabus recommends involving a wide variety of stakeholders for this reason. A review cannot prove the requirements defect-free, cannot measure performance and does not replace acceptance testing.

1. Think about who takes part in an early requirements review.
2. Recall that the review aligns their understanding and communication.
3. Rule out claims of proof, performance results or replaced testing.
4. Choose shared understanding.

Exam tip: Early reviews align stakeholders while changes are still cheap.

Static testing of documented requirements lets stakeholders make sure those requirements describe their actual needs.

Business representatives are the people who know what is actually needed, so their part in a requirements review is to confirm that the documented requirements reflect those needs. The other options describe checks that either belong to other participants (coding standards) or cannot happen before development (executed tests, performance results).

1. Notice the timing: nothing has been built yet.
2. Ask what business representatives uniquely know.
3. Recall that stakeholders validate requirements against their actual needs.
4. Choose confirming the requirements describe their needs.

Exam tip: Stakeholders in a requirements review check that it is the right thing to build.

White-box test cases depend on how the software is designed, so they can only be created after the design or implementation of the test object.

Branches only exist once there is a design or code to analyse, so branch tests cannot be created yet. Decision tables and boundary values are black-box techniques that need only the specified behavior, and a checklist can be prepared from experience before implementation.

1. Note what exists: approved requirements only.
2. Recall that black-box and experience-based tests do not need the code.
3. Recall that white-box tests need the design or implementation.
4. Choose the branch tests.

Exam tip: No design or code yet: no white-box tests yet.

Black-box test techniques are based on the specified behavior of the test object, so their test cases are independent of how the software is implemented.

The two black-box characteristics are their basis (specified behavior) and its consequence (independence from the implementation). Internal structure and needing the design first belong to white-box techniques, and strong dependence on tester skill belongs to experience-based techniques.

1. Assign each statement to the category it describes.
2. Keep the two that describe specified behavior and implementation independence.
3. Discard the white-box and experience-based characteristics.

Exam tip: Black-box: specified behavior in, implementation-independent tests out.

Test techniques help to develop a relatively small, but sufficient, set of test cases in a systematic way.

Since exhaustive testing is not possible, techniques give a systematic way to choose a manageable set of test cases that still covers what matters. Ordering by risk is prioritization, and techniques still rely on a test basis or on the tester's experience.

1. Recall that exhaustive testing is impossible.
2. Recall that techniques select a small but sufficient set systematically.
3. Reject options describing prioritization or removing the test basis.

Exam tip: Techniques give a small, sufficient, systematic set of tests.

Test techniques help the tester to define test conditions, identify coverage items and identify test data during test analysis and design.

Applying a technique such as equivalence partitioning yields test conditions (what to test), coverage items (such as partitions) and the test data needed. Each distractor includes one real item mixed with outputs of planning, management, execution or completion.

1. Recall the outputs of analysis and design.
2. Check each option for items produced in other activities.
3. Choose test conditions, coverage items and test data.

Exam tip: Techniques yield test conditions, coverage items and test data.

The syllabus's black-box techniques are equivalence partitioning, boundary value analysis, decision table testing and state transition testing; statement and branch testing are white-box; error guessing, exploratory and checklist-based testing are experience-based.

Equivalence partitioning, state transition testing and decision table testing derive tests from specified behavior. Branch testing derives them from the decisions in the code (white-box), and error guessing relies on the tester's knowledge of likely mistakes (experience-based).

1. Classify each technique by what it is derived from.
2. Mark branch testing as white-box.
3. Mark error guessing as experience-based.
4. Keep the remaining three.

Exam tip: EP, BVA, decision tables, state transitions: black-box.

The experience-based techniques in the syllabus are error guessing, exploratory testing and checklist-based testing.

Checklist-based testing uses a list of test conditions built from experience of what matters to users and of why software fails, so it is experience-based. Boundary value analysis and state transition testing are black-box; statement testing is white-box.

1. Recall the three experience-based techniques.
2. Classify the other options as black-box or white-box.
3. Choose checklist-based testing.

Exam tip: Error guessing, exploratory and checklist-based testing are experience-based.

Experience-based test techniques use the knowledge and experience of testers for the design and implementation of test cases.

The source of the tests is what the tester has seen go wrong before, which is knowledge and experience rather than a specification or the code. That makes it experience-based (error guessing is the typical example).

1. Identify the source of the tests: past failures the tester remembers.
2. Recognise this as tester knowledge and experience.
3. Choose experience-based.

Exam tip: Tests from the tester's past experience: experience-based.

Entry criteria are the preconditions for an activity; exit criteria are what must be achieved to declare it complete.

Entry criteria define the preconditions for undertaking an activity; if they are not met the activity is likely to be more difficult, time-consuming, costly and risky. Exit criteria define what must be achieved to declare the activity completed. Both are defined for each test level and differ by test objectives, and both are planning decisions rather than one group's property.

1. Recall the definition of each type of criterion.
2. Check whether each option limits the criteria to one activity, one scope or one group.
3. Keep the option that states what each criterion defines.

Exam tip: Entry = can we start? Exit = are we done? Both are defined per test level.

Exit criteria include measures of thoroughness and completion criteria.

Measures of thoroughness used as exit criteria include the achieved level of coverage, the number of unresolved defects, defect density and the number of failed test cases. An available test basis, assigned people and tools, and passed smoke tests are all conditions for starting, so they are entry criteria.

1. Decide whether each condition is checked before starting or before finishing.
2. Recall the thoroughness measures: coverage, unresolved defects, defect density, failed tests.
3. Pick the condition that measures how thorough the testing was.

Exam tip: Thoroughness exit criteria: coverage, unresolved defects, defect density, failed test cases.

Running out of time or budget can be a valid exit criterion if stakeholders accept the risk.

Exit criteria normally include measures of thoroughness such as coverage. Even when they are not satisfied, ending testing because time or budget has run out can be acceptable, but only if stakeholders have reviewed and accepted the risk of going live without further testing. Treating the deadline as automatically meeting the criteria skips that decision, and rewriting the target conceals the shortfall.

1. Note that the thoroughness criterion is not met.
2. Recall that running out of time or budget can be a valid reason to stop.
3. Add the condition: stakeholders must review and accept the residual risk.
4. Reject options that skip that decision or hide the shortfall.

Exam tip: Out of time is a valid exit only when stakeholders knowingly accept the risk.

Agile names: Definition of Done for exit criteria, Definition of Ready for a user story's entry criteria.

Exit criteria in Agile are often called the Definition of Done, which defines the team's objective metrics for a releasable item. Entry criteria a user story must fulfil to start development and/or testing are called the Definition of Ready. Acceptance criteria belong to an individual story.

1. Recognize that the question asks about exit criteria.
2. Recall the Agile names for entry and exit criteria.
3. Choose the name for the releasable-item exit criteria.

Exam tip: Ready = entry for a story; Done = exit for a releasable item.

Estimation based on ratios applies standard ratios from past projects to the current project.

With a development-to-test ratio of 5:2, test effort is two fifths of development effort: 750 x 2 / 5 = 300 person-days. Inverting the ratio gives 1875, adding both efforts gives 1050, and dividing by the sum of the ratio parts gives 214.

1. Write the ratio as development : test = 5 : 2.
2. Test effort = development effort x 2 / 5.
3. Compute 750 x 2 / 5 = 300.

Exam tip: Ratio estimate: test effort = development effort x (test part / development part).

Any tool that assists in testing is a test tool in the context of testing.

The CTFL syllabus ends its list of tool types with any other tool that assists in testing, and uses a spreadsheet as the example. What makes something a test tool is how it is used, not whether it was designed for testing or connected to other tools.

1. Look at how the spreadsheet is used: recording conditions and results.
2. Recall the syllabus rule that any tool assisting in testing is a test tool.
3. Reject conditions such as integration or purpose-built design.

Exam tip: A test tool is defined by its use in testing; a spreadsheet can be one.