CISM Security Control Implementation, Testing and Evaluation

This part of CISM domain 3B covers how approved security controls are put into operation, built into the organization's processes, and then tested and evaluated to show that they work and that they manage risk to an acceptable level. It picks up where control design stops. A control that has been selected on paper delivers nothing until it is implemented without breaking the business, embedded in the processes that would otherwise undo it, and checked with evidence strong enough for management to rely on.

The CISM view is a manager's view. The information security manager plans and sequences implementation, makes sure security is integrated into development, change and business operations, arranges testing that is independent and well evidenced, and tells management what the results mean. The manager does not own the business decisions the results feed into: a system owner decides whether a system goes live with a known flaw, and business management sets the risk appetite against which controls are judged. Many items in this area turn on that division of roles and on doing things in the right order.

Neighbouring material has its own lessons: security architecture, resources and frameworks (domain 3A, first part); policies, asset classification, control design and selection, and metrics (3A and the start of 3B); and awareness, external services and program reporting (rest of 3B). Risk acceptance and exceptions belong to risk management in domain 2.

Control implementation should be sequenced by the risk each control reduces, and rolled out in a way that limits disruption to operations. There is rarely enough budget or staff to implement every approved control at once, so the plan has to rank them. The ranking that serves the program is the size of the risk each control treats: the controls that close the largest exposures go first. Practical factors such as effort, dependencies and available skills still shape the schedule, but they decide how a high-priority control is delivered, not whether a low-risk one jumps the queue.

How a control is introduced matters as much as when. A change that reaches every user or every system at once, for example a new data loss prevention policy on outbound email, is usually safer when it starts small and widens in stages:

Before work starts, a sound plan also writes down what success will look like: the control objective the control must meet and the level of business impact that is tolerable. Those criteria become the yardstick for the review after go-live. Any change that could interrupt critical services also needs a back-out plan that has been tried, not just written.

Implementation is technical as well as procedural. A new control should be integrated with the existing architecture: its logs and alerts feed the security monitoring platform, its administrator access runs through the organization's identity and access management, and its configuration is recorded in the asset and configuration inventory. When systems or services are acquired rather than built, the same security requirements belong in the acquisition requirements and the contract, so the control is part of what is bought.

A control that will run on production systems should be tested first in an environment that represents production, because most deployment failures come from interactions with local configurations, integrations and older applications. Assurance from the supplier is about the product in general; it cannot speak for how the product behaves alongside this organization's own software, data and settings. Only a representative test answers that question.

A test environment never matches production perfectly. Where some systems cannot be reproduced, the gap is a known risk to be managed: the systems that were not tested are deliberately placed where a problem would be small and reversible, with a back-out route that has been rehearsed, so the first contact with production is itself a controlled test. Ignoring the gap, or waiting indefinitely for a perfect lab, are both ways of not managing it.

After go-live, a post-implementation review asks whether the control achieved what it was implemented for. Success is measured against the control objective and the success criteria set before work began: the risk the control targets has actually been reduced, and the business can still operate acceptably with it in place. Delivery measures describe the project, and activity measures describe workload; neither shows that the risk fell.

Suppose a privileged access vault is deployed to remove shared administrator passwords. A useful review checks whether shared passwords are actually gone and whether administrators can still do their jobs, not how many accounts were onboarded. If the review finds the objective has not been met, the next step is to diagnose why (scope, configuration, adoption, a change in the threat) before deciding on any remedy.

Security is integrated into organizational processes by building its requirements and checks into the processes themselves, so that controls are designed in and are not undone later. The two processes that matter most here are system development and change management.

The system development life cycle

Security requirements are cheapest to meet when they are defined in the requirements phase, alongside the business requirements, because the design then accounts for them from the start. Each later phase in which a security need is discovered turns it into rework, and the cost of that rework grows with every phase that has already been built on the flawed design.

Fast, iterative delivery changes how this works. When security is a checkpoint outside the delivery teams' rhythm, it becomes a queue, and teams under pressure learn to route around it. The durable fix is to move security inside the rhythm: security acceptance criteria are written into the user stories, automated checks run in the build pipeline on every iteration, and the team's definition of done includes them. Assurance then arrives continuously with the work rather than in a single review at the end.

Change management

Hardened configurations are eroded by ordinary changes made without regard to security. Finding and correcting the drift afterwards is detective and lets it happen again. The preventive approach is to have change requests assessed for their effect on security before they are approved, so a change that would weaken a control is caught in the approval step. Responsibility for operating systems stays with operations; security's role is to shape the process and oversee it, which also preserves segregation of duties.

Change management must also cover emergency changes. When a change is made outside the normal process to restore a critical service, reversing it would bring the outage back. The control is a retrospective review: the change is assessed for security impact, documented and approved after the fact, and corrected if needed. The emergency procedure should define who may authorize such changes and the deadline for the retrospective review.

Security becomes part of business operations when business units carry defined security responsibilities in their own processes and roles, and when controls are tuned to legitimate business needs. Awareness alone does not make a unit act; accountability does. When specific security steps, such as approving access to the unit's systems or handling its sensitive records, are owned by named roles within the unit's own workflows, security work happens where the business work happens instead of being left to IT.

A control that obstructs legitimate work invites workarounds, and the workaround is usually a symptom of a design that ignored a business need. Imagine removable-media blocking that also stops field engineers from loading firmware onto equipment, and engineers starting to move files through unapproved file-sharing sites. The productive response is to revisit the control with the people who own that work, find a controlled way to meet the genuine need, and keep the protection for everything else.

Security testing as an input to go-live

Security test results feed release and go-live decisions, but those decisions belong to the business owner of the system, who weighs the security risk against the cost of delay. The information security manager's job is to make sure that decision is informed, timely and taken by the right person:

• Agree release criteria in advance with business owners: which kinds of finding prevent release, which can go live with a dated fix and compensating measures, and who signs off. Criteria settled beforehand make security results a predictable input rather than a negotiation each time.
• When a serious finding appears close to launch, the owner receives the risk in business terms, the options for remediation and any temporary measures, and makes the call. The decision and the accepted risk are then recorded so they have an owner.

Formal acceptance of a residual risk follows the organization's risk acceptance process, which is covered with risk management in domain 2.

Design effectiveness asks whether a control, performed as intended, would achieve its objective; operating effectiveness asks whether it was actually performed as designed, consistently, throughout the period under review. They need different evidence, and a control can pass one test and fail the other.

Take a quarterly recertification of firewall rules. Walking one cycle through the procedure shows that, if performed, it would remove rules nobody still needs: the design is sound. Inspecting the past year then shows two quarters with no recertification at all. The conclusion is about operation, not design, so the fix lies in ownership, scheduling and follow-up of the existing procedure.

Even a control that is well designed and operates every time may not reduce risk enough. That is the adequacy question covered in the last section.

Control testing techniques differ in how directly they show that a control works: reperformance and inspection of evidence are strongest, observation is point-in-time, and inquiry alone is the weakest.

A document that requires a setting, or a ticket that approved it, shows intent or authorization; it does not show the setting is in force on the live system. When a control owner asserts that a control runs but cannot produce records, the tester looks for the traces the control would leave if it ran, such as its outputs or the actions it should have triggered. If those traces do not exist, the control cannot be shown to operate, and the lack of evidence is itself part of the finding.

Independence

Assurance is strongest when the tester has no stake in the result, which means the person testing is not the person who runs, configured or sold the control. Independence is a matter of degree rather than a requirement to hire an outside firm: within a small team it can come from separating who builds a control from who tests it, and from having someone outside the team review the results. What gives little assurance is a control's operators grading their own work, however the result is later packaged.

Test populations, sampling and exceptions

A sample supports a conclusion only about the population it was drawn from, so the population must be complete, obtained independently of the control operator, and span the whole period the conclusion covers.

• Independent source. To test that new privileged accounts were approved, the list of accounts should come from the directory's own creation records, not from a list prepared by the team that creates them. The unapproved accounts are exactly the ones a self-prepared list might leave out, and a well-sized sample from a biased population is still biased.
• Whole period. A conclusion about a full year needs items from the full year. A quarterly statement about a weekly control needs samples spread across all of that quarter's weeks.
• Completeness. Before sampling, the tester reconciles the population to another source, such as change records to deployment logs, to confirm nothing is missing.

When testing turns up an exception, the tester's next step is to understand it: what caused it and whether it is a one-off or a sign of something systematic. That analysis drives everything after it, including whether to widen testing, how to rate the control and what remediation to recommend.

How often a control is tested should be set mainly by the risk it mitigates and by how often the control or its environment changes. Testing exists to catch control failures before they cause harm, and the chance and cost of that harm depend on risk and change. Resourcing and scheduling decide how the tests are delivered, not how often a control needs them.

A risk-based schedule assumes the environment stays broadly stable. A major change, such as a merger that connects a new business unit to the network, replacement of the identity platform or a reorganization of the people who run controls, makes the last test result unreliable. The next test should then happen soon after the change, covering the controls the change could plausibly have affected, rather than waiting for its scheduled date.

Continuous control monitoring

Continuous automated monitoring compares the actual state of controls with a baseline as it changes and alerts on deviation. It suits automated, configuration-based controls whose state tools can read, such as firewall rule sets, logging settings on servers or patch levels reported by endpoint management, especially where they change often. These controls can drift at any time, so evidence collected once a year says little about the months in between. Manual, periodic or judgement-based controls (an access review, an approval) produce occasional evidence and are tested by sampling instead.

Continuous monitoring does not replace independent testing. It can only report on what its rules check, so a new type of asset, or a control no rule examines, simply never appears in the results. Someone independent should periodically review what the monitoring covers and whether its rules are right, so the assurance it gives stays complete.

Vulnerability scanning identifies known weaknesses broadly and repeatedly; penetration testing actively exploits weaknesses, alone or in combination, to show whether a specific asset can be reached. They answer different questions, and the information security manager chooses, authorizes and interprets them.

A scan that logs in to each system (a credentialed or authenticated scan) can read what is installed, so it finds missing patches and local misconfigurations that a scan from the network cannot see. Neither kind of scan proves whether a weakness can be exploited; that is what penetration testing adds.

Authorization, rules of engagement and scope

Before any penetration test, especially against production, management must approve it in writing and the parties must document the rules of engagement: what is in scope, when testing may happen, which techniques are allowed, who to contact and how to halt the test if something goes wrong. That approval is what separates a test from an attack; other preparations support the test but do not make it legitimate or set its limits.

Results must be reported within their limits. Suppose a three-day test of a mobile app's API finds nothing the testers could exploit. That tells management the testers found no way in through that interface in that time; it says nothing about the systems they were not asked to test or the attacks they had no time to try. Reporting it as proof that the organization is secure overstates it. Findings from scans and tests flow into the vulnerability management process, where they are prioritized by risk and tracked to closure.

When testing finds a deficiency, the first step is to assess the risk exposure it has created; when deficiencies recur, the cause must be found and fixed; and even fully effective controls must still be judged against the organization's risk appetite.

Handling a deficiency

A deficiency matters in proportion to the risk it creates. If testing finds that audit logging has been switched off on a group of database servers, the manager first establishes which systems, what data they hold, for how long the gap existed and whether anything happened in that window. That assessment is what decides urgency, who needs to know and whether interim measures are needed; responses chosen before it are guesses.

When the same kind of finding comes back test after test, fixing each instance is treating the symptom. The manager should find out why the control keeps failing, for example unclear ownership, an incomplete inventory or an automated job that silently fails, and fix that.

Remediation is confirmed by retesting, not by a ticket being closed. When retesting capacity is limited, the highest-risk items are verified first, and reports distinguish clearly between remediation that has been verified and remediation that has only been reported, so management is not given false comfort.

Effective is not the same as adequate

The CISM outline expects the manager to evaluate whether controls are appropriate and manage risk to an acceptable level, which goes beyond whether they operate. Three situations come up repeatedly:

Retesting answers whether controls run; it does not answer whether they are enough. Risk appetite is set by senior management, and treating residual risk that exceeds it, or formally accepting it, goes through the risk management process.