Maintenance Testing in CTFL: Categories, Triggers and Scope

Maintenance testing is the testing of changes to a system that is already operational — software that is in production and being used. The syllabus defines it by the situation rather than as a test level of its own (in common ISTQB usage it can involve any test level and test type): it is defined by when it happens (after release) and why (something about a live system has changed).

The change does not have to be in the code. Maintenance testing is needed whenever an operational system or its environment changes: a code fix, a new feature in a scheduled release, new infrastructure or a new version of the platform underneath the application, a move to another platform, or the system being retired. Testing of software that has never been released — for example, system testing of a new product before its first launch — is not maintenance testing, even if it happens late in a project.

Compared with testing during initial development, maintenance testing has two jobs at once (covered below): confirm that the change itself works, and check that the large, unchanged rest of the live system still works.

The CTFL syllabus, referring to ISO/IEC 14764, groups maintenance by the reason for the change. The reason, not the size or urgency of the change, decides the category.

Two distinctions are easy to blur. Corrective vs adaptive: corrective needs a defect; adaptive needs an environment change. A browser withdrawing a feature the application relied on is not a defect in the application, so reworking the application to keep working in that browser is adaptive. Performance vs maintainability: both improve a system that already works, but performance targets speed or resource use, while maintainability targets the cost and effort of future change (often with behaviour unchanged).

Maintenance reaches production in two ways: planned releases/deployments and unplanned releases/deployments.

• Planned (release-based): changes are bundled and shipped on a known schedule — for example, enhancements every quarter, or a defect fix that waits for the next scheduled release.
• Unplanned (hot fixes): a change is deployed outside the normal schedule because it cannot wait — typically a severe production failure or an urgent security vulnerability fixed and deployed within hours.

Planned vs unplanned describes how the change is delivered; it is independent of the maintenance category. A corrective change can ship as a hot fix or in the next scheduled release. Hot fixes are usually delivered under time pressure. The syllabus does not name urgency as a scope factor, so a reasonable reading is that the scope is still set by the risk of the change and the system around it (see the scope factors below), even when the schedule is tight.

Impact analysis identifies which parts of a system could be affected by a change, and it may be done before the change is made to help decide whether the change should be made at all, based on its potential consequences in other areas of the system.

That timing matters. Impact analysis is not only a post-change activity to pick regression tests; it is input to a decision. If a proposed change touches a module that many other parts reuse — for example a currency-rounding routine that quotes, customer statements and an outbound file feed all call — the analysis shows the change has wide consequences. The business can then weigh the benefit against the risk and the cost of testing before approving it.

When the change goes ahead, the same result also informs testing: a change with a wide impact means a wider maintenance testing scope, with more regression testing across the affected areas, than a change confined to one isolated module.

Testing a change to a system in production evaluates two things:

1. The success of the implementation of the change — does the fix, enhancement or adaptation do what it was meant to do?
2. Possible regressions in the parts of the system that remain unchanged — which is usually most of the system.

The second point is why maintenance testing reaches beyond the modules that were edited. A change to two modules can break callers, shared data, interfaces or configurations elsewhere; the unchanged parts are what users relied on yesterday, and a live system's users notice when they stop working. A quick test of only the changed lines therefore checks the first point and ignores the second.

(How confirmation testing and regression testing differ in general, and why regression tests are good automation candidates, belongs to the test levels and test types lesson.)

The scope of maintenance testing typically depends on three factors:

• The degree of risk of the change — how much harm a failure in the changed area could cause. Rewording an internal tooltip carries little risk; altering the dose calculation in a hospital prescribing system carries a lot.
• The size of the existing system — a larger, more interconnected system has more unchanged parts in which a regression could appear.
• The size of the change — a larger change has more to confirm and more ways to affect other parts.

No single factor decides alone: the three combine, so a small change can still need wide testing when it sits in high-risk logic, and a large change in a low-risk, isolated area may need less.

The events that trigger maintenance and maintenance testing fall into three groups:

Note that corrective changes and hot fixes are modifications: classify by what happened to the live system. A change to the system's software is a modification whatever its reason or delivery speed.

An upgrade or migration of the operational environment can require two kinds of tests: tests of the new environment (does the platform, OS or infrastructure work as expected?) and tests of the changed software (if the application was adapted or partly rewritten to fit the new environment, does that code work?).

Data conversion tests are tied to a specific situation: data from another application is migrated into the system being maintained — for example, a city library service taking member records from a neighbouring library's separate catalogue application into the lending system it maintains. The syllabus ties data conversion tests to that cross-application case; a reasonable reading is that moving a system's own data along with it to a new platform belongs to the environment migration rather than being a data conversion in this sense.

A purely environmental change still counts. Replacing the hardware a live warehouse-management system runs on, with no change to its features, is an operational environment change — and testing it is maintenance testing.

Retirement — an application reaching the end of its life — is a maintenance trigger even though the system is being switched off. Two kinds of testing may be needed:

• Data archiving tests when long data-retention periods are required, for example records that must be kept for many years for legal or regulatory reasons. They check that the data is archived completely and correctly.
• Restore and retrieval procedure tests after archiving, in case some data is needed during the archiving period — for example when a legal dispute needs one specific historical record. Archiving that nobody can read back is not useful.

Retirement often comes with migration. When one system is switched off and its live data moves elsewhere, two triggers occur together and point at different systems: data migrated into a maintained system from the retiring application is a migration concern (data conversion tests on the receiving side), while data that must be kept but not moved is a retirement concern (archiving, and possibly restore and retrieval tests).

Retirement concerns the system being maintained. A change to something underneath it, such as its platform or a component it depends on, is an environment trigger for that system, not its retirement.