CCSP: Evaluating Cloud Service Providers, Common Criteria and FIPS 140

You evaluate a cloud service provider (CSP) by defining your criteria first and then verifying each candidate against them with evidence. The criteria come from the organisation: the regulations that apply to the data, its classification, the risk appetite, and business needs such as recovery times, data residency and integration. Certifications, attestations and questionnaires are evidence for those criteria; they are never the criteria themselves.

The order matters because evidence collected without criteria cannot be judged. A pile of certificates tells you what a provider chose to have audited, not whether it covers what your workload needs. Discovery and classification come before protection, and assessment comes before contracting.

1. Understand the workload and data: what is stored or processed, its classification, which laws and industry rules apply (for example GDPR, HIPAA, PCI DSS).
2. Set the criteria, separating mandatory (pass/fail) from weighted (scored) criteria.
3. Collect evidence: organisational certifications and reports, product certifications for key components, the provider's documentation and contract terms.
4. Verify the evidence: is it independent, genuine, current, and does its scope cover the services and locations you will use?
5. Record gaps and decide how each is closed: other evidence, your own testing, compensating controls, or rejecting the candidate.

A mandatory criterion is not traded off. If policy requires that encryption keys stay under the customer's exclusive control and a candidate offers only provider-managed keys, the candidate fails that criterion however many certificates it holds; weighing extra certifications against a must-have turns it into a nice-to-have. Likewise, when no report has tested something you require, the criterion is not relaxed because the evidence is silent: you obtain other evidence that does test it, such as an exercise you observe or perform yourself.

Contract terms (right to audit, SLAs) and the audit process itself are covered in Domain 6; this lesson is about judging the evidence a provider offers.

CSP evidence comes in two kinds. Organisational certifications and attestations (ISO/IEC 27001, SOC 2, CSA STAR) give assurance about how the provider runs its service: governance, change management, administrator access, data centre operations. System/subsystem product certifications (Common Criteria, FIPS 140) give assurance about a component: that a hypervisor, firewall, storage array or cryptographic module does what it claims in an evaluated configuration.

A complete evaluation usually needs both. When a requirement has a product part (is this component evaluated or validated?) and an operational part (did the provider run the related processes as designed over time?), it is two questions: the product part is answered by a product certificate such as Common Criteria or FIPS 140-3, the operational part by organisational evidence such as a SOC 2 Type II report whose tested controls cover that process. Product certificates cannot answer operational questions, and raising a product's assurance level does not change that.

A certificate or report covers only its stated scope, so the first verification is whether the services, locations and activities you will use are inside it. An ISO/IEC 27001 certificate certifies an ISMS within the scope defined under clause 4.3; anything added or left outside that scope (a business unit, an acquired facility, a product line) has not been audited even though the provider is "ISO certified".

Read three things:

• The scope statement on the certificate: which services, sites and activities. A scope limited to a vendor's own software engineering and customer support functions says nothing about the infrastructure a different company runs for it; for assurance about that infrastructure you need the other company's own independent evidence.
• The Statement of Applicability (SoA) (clause 6.1.3): which Annex A controls are included and which are excluded, with justification. A justified exclusion does not invalidate the certificate, but an excluded control is not covered by it. If your criteria need that control, obtain other independent evidence.
• Subservice organisations and carve-outs: where the provider relies on another provider, the certificate or report may exclude it. ISO/IEC 27001:2022 Annex A control 5.23 (information security for use of cloud services) covers how the certified organisation acquires, uses, manages and exits the cloud services it consumes, and controls 5.19 to 5.22 cover supplier relationships in general. Neither is an audit of those suppliers: they show the certified organisation manages its suppliers, not that the suppliers are secure.

The same rule applies to every kind of evidence: a PCI DSS Attestation of Compliance lists the services assessed, a SOC 2 report has a system description, a Common Criteria certificate names the Target of Evaluation, and a FIPS certificate names a module version.

Evidence is only useful if it is still in force, really issued, and tested by an independent party. A PDF shows what was true when it was issued; it cannot show a later suspension or withdrawal.

• Confirm status at the source. ISO/IEC 27001 certificates run on a three-year cycle with surveillance audits, and can be suspended or withdrawn mid-cycle. Check with the issuing certification body or the register maintained under its accreditation body, not by reading the printed expiry date or asking the provider to sign a statement. Do the same for FIPS 140 (NIST's CMVP lists) and Common Criteria (the scheme's or the Common Criteria portal's records).
• Check who issued it. ISO publishes the standard but issues no certificates and keeps no register of certified organisations. Certificates are issued by certification bodies, and an accredited certification is one whose certification body is itself accredited for that standard by a recognised accreditation body (a national body that is a signatory to the International Accreditation Forum's Multilateral Recognition Arrangement, the IAF MLA). A certificate from an unaccredited body can be real and current yet still fail a criterion that requires accredited certification: a certificate's status and its issuer's accreditation are separate checks, and each needs its own source. Check the certification body in its accreditation body's records (or the IAF CertSearch database where it participates).
• Check the edition. The International Accreditation Forum's transition from ISO/IEC 27001:2013 to ISO/IEC 27001:2022 ended on 31 October 2025. A 2013-edition certificate is no longer valid after that date, whatever its original cycle; ask for the 2022-edition certificate.
• Check the report period. A SOC 2 Type II report gives an opinion on a past period. A bridge letter (gap letter) signed by the provider's management covering the months since is a management assertion with no auditor testing behind it: weigh it as such and look for the next report. It neither extends the opinion nor voids the report.
• Separate described from tested. In a SOC 2 report, the system description says what the provider does; only the controls in the auditor's tests of controls carry tested assurance. A process that is described but not tested is still an assertion.
• Prefer independence. Self-assessments (a CAIQ, a provider's own questionnaire, a trust-centre compliance statement) are useful for discovery but do not meet a criterion that requires independently verified assurance.

ISO/IEC 27001 is the certifiable standard: it specifies the information security management system (ISMS). ISO/IEC 27017 and ISO/IEC 27018 are cloud codes of practice that extend ISO/IEC 27002's control guidance; providers are assessed against them as an extension within an ISO/IEC 27001 certification, so a claim to be "27018 certified" is verified by checking that the ISO/IEC 27001 certificate references 27018 and covers the service you use.

A cloud standard does not satisfy a criterion it was not written for. ISO/IEC 27018 addresses the location of PII by requiring the processor to disclose where PII may be processed; that disclosure tells you where a provider may process PII, so you can compare it with your residency rule; it is not itself a control that meets one. Nor does 27018 certify GDPR compliance.

The Cloud Security Alliance's STAR programme publishes provider assurance in the public STAR Registry, built on the Cloud Controls Matrix (CCM). Its levels separate what a provider says about itself from what an independent party has checked.

Appearing in the same registry does not make two entries equivalent. A Level 1 entry is the provider grading itself however complete or favourable its answers, so it cannot meet a policy that requires independent verification. A Level 2 entry can, even when the audit raised findings: findings with corrective actions are a normal audit outcome, and the evaluator reviews them against the criteria rather than treating any finding as disqualifying. Independence and CCM coverage must come from the same assessment. A plain ISO/IEC 27001 certificate is independent, but its auditor tested the ISMS and Annex A controls, not the CCM control objectives; a CAIQ covers the CCM, but only as the provider's own answers. Combining separate pieces of evidence does not produce an independent assessment against the CCM: that needs STAR Level 2, where the CCM is added to the audit or examination itself, or an equivalent independent CCM assessment. The quick way to tell the Level 2 routes apart is the base engagement: ISO/IEC 27001 audit means Certification, SOC 2 examination by a CPA firm means Attestation.

To verify a provider that will store, process or transmit cardholder data, obtain its service provider Attestation of Compliance (AOC) and its responsibility matrix for the specific services you will use. An ISO/IEC 27001 certificate, a SOC 2 report or a "PCI compliant" statement in a trust centre is not PCI DSS evidence.

PCI DSS v4.0.1 puts the obligations on both sides:

• Requirement 12.8 (the customer): keep a list of third-party service providers, have written agreements in which they acknowledge responsibility for the account data they handle, perform due diligence before engagement, monitor each provider's PCI DSS compliance status at least once every 12 months, and maintain information about which requirements the provider manages, which the customer manages, and which are shared.
• Requirement 12.9 (the service provider): acknowledge its responsibility in writing and supply customers with its compliance status and responsibility information on request.

The AOC and the responsibility matrix answer different questions. The AOC records the provider's own assessment of the services it lists; it does not divide requirements between the provider and its customers. That split is what Requirement 12.8.5 asks the customer to maintain and Requirement 12.9.2 asks the provider to support. If a provider will not document it, make reliance on the provider conditional on getting that information: do not assume the provider manages every requirement in its assessed scope, and do not assume the customer owns them all.

Two further consequences are frequently missed. First, verification is not a one-time onboarding step: an AOC older than the 12-month monitoring cycle no longer evidences current status. Second, the AOC covers only the services it lists as assessed. A service released after the provider's last assessment, or simply not listed, is unassessed; PCI DSS allows such a service to be reviewed as part of each customer's own assessment until the provider's next assessment includes it.

A provider's certification covers the controls the provider operates; the customer still owns its share, and that share grows as you move from SaaS to PaaS to IaaS. A provider's PCI DSS AOC does not make an IaaS workload compliant: the requirements the matrix assigns to the customer (for example firewall rules it writes and anti-malware on its own servers) stay with the customer. The provider's responsibility matrix is the document that maps each requirement to the provider, the customer or both, so it is where a customer's own share is identified.

Complementary user entity controls (CUECs) in a SOC 2 report state the controls the report assumes the customer operates, such as quarterly review of user access. The auditor did not test them, and if the customer does not perform them the assurance the report suggests does not hold.

When an auditor asks for evidence of a control the customer configures (for example the access policies or backup schedules it sets in the provider's console), the evidence is the customer's own: its configuration and records showing the control operated. Provider certificates cover only the provider's part, and a policy states intent rather than proving operation.

Common Criteria (ISO/IEC 15408) is the international framework for evaluating the security of IT products; under the Common Criteria Recognition Arrangement (CCRA, 2014), certificates from participating schemes are mutually recognised only up to EAL2 (plus ALC_FLR) or for conformance to a collaborative Protection Profile; higher levels are recognised through regional arrangements such as SOG-IS and the EU's EUCC scheme. Its vocabulary is precise:

An EAL measures the depth and rigour of the evaluation, not the strength or number of security functions. The levels run from EAL1 (functionally tested), EAL2 (structurally tested), EAL3 (methodically tested and checked), EAL4 (methodically designed, tested and reviewed), EAL5 (semiformally designed and tested), EAL6 (semiformally verified design and tested) to EAL7 (formally verified design and tested). A product certified at a higher EAL may claim fewer or different functions than one at a lower EAL, so when comparing products against a required function list, read each Security Target first; PP conformance helps only if the profile matches your needs.

The "+" means augmented: assurance components added to the package. Flaw remediation (ALC_FLR) is the most common augmentation; its components are ALC_FLR.1 basic flaw remediation, ALC_FLR.2 flaw reporting procedures and ALC_FLR.3 systematic flaw remediation, so "EAL2 augmented with ALC_FLR.3" means the evaluation also examined how the developer systematically tracks and fixes reported security flaws. Augmentation adds assurance about the developer's processes; it adds no security functions.

What a Common Criteria certificate does and does not cover

A Common Criteria certificate applies only to the evaluated version in the evaluated configuration, under the environment the Security Target assumes. Anything else is outside its assurance:

• Other versions do not inherit it. When the developer changes a certified product, the scheme's assurance continuity process decides: minor changes can be accepted in a maintenance report that names the new version; significant ones need re-evaluation. Verify a vendor's "still certified" claim by finding that report on the scheme's or the Common Criteria portal's records.
• Functions beyond the Protection Profile are covered only if the Security Target claims them and they sit inside the TOE boundary. A PP sets a floor: a Security Target may claim conformance to a PP and add further security functions, which are then evaluated too. PP conformance on its own says nothing about those extras, and a function the ST does not claim, or that lies outside the TOE, was not evaluated however the product is marketed. To rely on a specific function, find it in the ST's TOE description and security functional requirements.
• Features outside the ST (such as enabling remote management the ST excludes) take the deployment out of the evaluated configuration.
• Environment assumptions are conditions, not advice. If the ST assumes the product is managed only from a dedicated management network, exposing its administration interface to the internet leaves the evaluated environment. The certificate is still genuine; it simply gives no assurance for that use until the gap is addressed with other controls or evidence.
• Services built on a certified product do not become certified. A marketing line such as "uses certified components" may describe a different model, version or configuration, so the first check is that the certified TOE matches what protects your data; recognition and EAL matter only after that.

Organisational evidence (an ISO/IEC 27001 certificate, a SOC 2 report) never substitutes for these checks: it does not record product evaluation facts.

FIPS 140 is the US federal standard for cryptographic modules, validated jointly with Canada through the CMVP; FIPS 140-3, which aligns with ISO/IEC 19790, is current and FIPS 140-2 is historical. It defines four increasing security levels:

Choose the lowest level that meets every requirement: find the strongest physical-security and authentication requirement you have, and pick the first level that provides both. A higher level also satisfies them, but adds protection, cost and constraints beyond what was asked.

Levels are assigned per area of the module (physical security, authentication and others), and the overall level is the lowest of them. The overall figure can therefore understate a module's strength in one area and say nothing about which area holds it back, so when a single area matters, read the per-area table in the module's security policy.

Verifying a FIPS 140 validation claim

A module is validated only when the Cryptographic Module Validation Program (CMVP), run by NIST and the Canadian Centre for Cyber Security, has issued a certificate naming that module and its hardware, firmware or software versions. Verify a claim by looking the certificate up on NIST's validated modules list and matching the name and version in use.

• "Compliant", "designed to meet" or "uses approved algorithms (AES-256)" is not validation. Neither is a mention in an ISO/IEC 27001 SoA or a SOC 2 report.
• Versions matter. A certificate covers the versions it names, and a later firmware or software release is not covered until a certificate names it. A module on the Modules In Process list has been submitted and is under review: that is not validation. A module is validated only once the CMVP has issued its certificate.
• Approved mode. The module's security policy defines its approved mode and approved services. Cryptography an application performs with non-approved algorithms is outside the validation, even inside a validated library; in IaaS, configuring this correctly is the customer's job. A validation covers the module named on the certificate, not the system built around it.
• FIPS 140-2 is historical. New FIPS 140-2 submissions closed in 2021-2022, and in September 2026 all remaining FIPS 140-2 certificates moved to the CMVP historical list, regardless of their issue date (certificates otherwise sit on the active list for five years). Historical modules may keep running in existing systems under the owner's risk decision, but a new system should rely on a FIPS 140-3 validation. A 140-2 certificate does not convert to 140-3.

The exam outline still writes "FIPS 140-2"; the concepts (levels, validated modules, approved mode) carry over to 140-3.