CCSP: Cloud Software Assurance and Security Testing (SAST, DAST, IAST, SCA)

Cloud software assurance is the justified confidence that an application works as intended and is free of exploitable weaknesses; validation is the testing and review that produces the evidence for that confidence. CCSP sub-domain 4.4 groups four activities under it: functional and non-functional testing (increasingly automated in CI/CD pipelines), security testing methodologies (black-box, white-box, SAST, DAST, IAST and SCA), quality assurance, and abuse case testing.

Two words are worth separating. Verification asks "did we build the product right?" — does the code meet its specification and the organisation's standards. Validation asks "did we build the right product?" — does it meet the business and security requirements in the environment where it will run. Testing supplies evidence for both, and quality assurance checks that the evidence was produced by the process the organisation defined.

The cloud changes the work in three ways. Releases are frequent, so tests must be automated and run inside the delivery pipeline rather than in a final phase. Environments are cheap to create, so a production-like staging environment is a realistic expectation, but it must not be filled with real personal data. And part of the stack belongs to the provider, so the customer may test only its own side of the shared-responsibility line and must rely on the provider's attestations for the rest. The secure SDLC phases themselves are covered in the lessons on describing and applying the secure SDLC; this lesson is about proving the software is fit to release.

Functional testing checks that each feature does what its requirement says; non-functional testing checks how well the system behaves while doing it — performance, scalability, availability, resilience, usability and security. In a library-loans app, tests confirming that a borrower can renew a book, that an overdue item accrues the correct fine, or that a returned item shows as available again are functional: each compares an output with an expected result. Checking that catalogue searches still return within their target time while thousands of simulated sessions browse at once is non-functional: the feature works, and the question is whether it keeps working under realistic conditions.

Each test type closes one gap and leaves others open, so the useful question is always which failure a test would actually have exercised. Functional tests prove behaviour at whatever volume the test happens to generate, which is usually tiny; only a load test at the expected peak shows whether capacity holds, so an exam-registration portal that passes every functional check can still collapse the morning registration opens. Load testing in turn says nothing about losing a component, because nothing is lost. Evidence of resilience comes from exercising the failure, not from contract terms or documentation: a promise to keep running through the loss of a site is demonstrated only by taking a site offline in an environment that genuinely has more than one. Availability and resilience are classic non-functional requirements, which is why the CCSP treats them as part of software assurance and not only of operations.

In continuous integration and continuous delivery (CI/CD), every change triggers an automated pipeline that builds the software, runs its tests and promotes the build through environments only when the tests pass, so testing becomes a repeated gate instead of a phase at the end. Continuous integration merges developers' changes frequently into a shared branch and tests each merge; continuous delivery keeps every passing build ready to release, and continuous deployment releases it automatically.

Regression testing is the backbone of the pipeline: the automated suite reruns after every merge to confirm that nothing which already passed has broken. It is different from a smoke test (a quick check that the build runs at all), from acceptance testing (the business confirming requirements are met) and from penetration testing (a skilled attempt to exploit the system).

Two practical points come up repeatedly:

• Turn every fixed security flaw into a regression test. A fix without a test is protected only by memory. Suppose a team fixes a flaw where a session token stayed valid after the user signed out; if no test checks for it, a later refactor of the session code can bring it back unnoticed. An automated test that replays the attack and expects it to fail runs on every build, so the regression is stopped at the merge that would reintroduce it. Scheduled reviews add value, but the gap between them is the window in which a returned flaw goes unnoticed.
• Keep feedback fast without losing coverage. A suite that takes hours creates a tension: run it rarely and broken changes linger on the shared branch; run all of it on every change and delivery stalls. Test impact analysis resolves it by running, on each merge, the tests affected by the files that changed plus a smoke set, and running the full suite on a schedule or before release. Every test still runs; what changes is when.

Pipeline tests are only as trustworthy as the pipeline itself: build definitions, test results and gate decisions should be version-controlled and logged, so that a release can be traced to the evidence that allowed it. Configuration management and versioning are taught in the lesson on applying the secure SDLC.

The "box" describes how much the tester knows: black-box testers start with no internal knowledge, white-box testers have full access to code, design and credentials, and gray-box testers have partial knowledge, typically a normal user account and some documentation. The choice is a trade-off between realism and coverage.

Match the box to the question being asked. When the worry is what an authenticated user could do beyond their rights, give the tester the same kind of access that user has, ideally two such accounts so one can be used to attack the other. In a university coursework system, for example, a tester holding two student logins checks whether one student can open, change or download the other's submissions. When the aim is coverage per unit of effort rather than realism, hand the testers everything: time not spent rediscovering what the organisation already knows is spent finding flaws. When the aim is to measure what an anonymous outsider can reach, black-box is the honest model. Static analysis tools are white-box by nature (they read the code); DAST is black-box (it sees only the running application); IAST sits in between.

In the cloud the box is sometimes chosen for you. A customer assessing a hosted application it does not own has only the interfaces the vendor chooses to expose, typically a non-production endpoint and some accounts. Code-based techniques need the code, so in that position the customer is limited to dynamic, black-box or gray-box testing.

SAST reads the code without running it, DAST attacks the running application from outside without seeing the code, and IAST places an agent inside the running application to watch the code paths that tests exercise. Each sees a different part of the problem, so mature programmes use them together.

Points worth reasoning through:

• SAST belongs early because it does not need a deployed application. That, not accuracy, is its advantage: it reports more false positives than methods that run the code, because it cannot tell whether a suspicious path is ever reached. Its blind spot follows from the same fact. Anything decided at deployment or run time, such as the TLS versions a load balancer accepts or an over-permissive storage policy set in the environment, is invisible to a tool that reads only application source code (infrastructure-as-code scanning can catch such settings when they are defined in templates).
• DAST needs a target that can safely be damaged. Authenticated dynamic scans write data as they probe, so the target must be both representative of production and expendable: a production-like staging environment holding masked data. An environment that differs from the deployed configuration gives misleading results; an environment holding real customers' data or serving them puts both at risk.
• A dynamic scanner can only test what it can reach. If most of an application sits behind sign-in and the scanner has no way in, a short or clean report measures the scanner's reach, not the application's security. Give it test accounts for each role and confirm in its logs that it crawled the authenticated pages.
• IAST's coverage equals the tests' coverage. It adds little pipeline time because it rides along with the functional suite the team already runs (the agent does add some runtime overhead to the instrumented application), and it ties each runtime finding to a code line, which cuts the false-positive noise teams complain about with external scanners. Code that no test exercises is never observed, so gaps in the functional suite become gaps in IAST; targeted test cases or manual testing must cover them.

Penetration testing is a manual, goal-driven exercise by skilled testers that chains weaknesses together; automated SAST, DAST and IAST scale across every build but do not replace human judgement on business logic.

Software composition analysis (SCA) identifies the open-source and third-party components in an application — including transitive dependencies pulled in by other libraries — and checks each against databases of known vulnerabilities and against its licence terms. Most of a modern cloud application's code is not written by the team, so SAST of first-party code leaves most of the attack surface unexamined.

What SCA gives you:

• An inventory. A list of every component and version in a build, often exported as a software bill of materials (SBOM). Declared top-level libraries are not enough: the vulnerable component is often a dependency of a dependency.
• Known-vulnerability matching. Components are matched to public advisories (CVE identifiers and similar), and the build can be failed when a severe one is present.
• Licence compliance. Strong copyleft licences (such as the GNU GPL) can oblige a company that distributes a combined or derivative work to make its corresponding source code available to the people who receive it, and network-copyleft licences (such as the GNU AGPL) extend the obligation to a modified version that users interact with remotely over a network, which matters for SaaS where nothing is ever distributed. Licence terms are a property of the components, so the component inventory is where this risk surfaces; SCA flags such licences for legal review.

Keep a component record for each release that reaches an environment. When a critical vulnerability is announced in a popular library, the urgent question is which running services contain the affected version, and what is running is whatever build was deployed, which may be far older than the repository's current code. An inventory captured at build time and linked to deployment records answers that question directly and quickly; a fresh scan of today's source answers a different one.

When a finding has no fix. Sometimes an SCA gate blocks a release for a severe vulnerability in a dependency that has no patched version. The principle is that an exception should be a governed decision, not a workaround: the gate and the finding stay intact, and the accountable risk owner formally accepts the risk for a limited time. A sound acceptance documents how exposed the vulnerable code actually is, applies a compensating control (restrict, isolate or monitor the path that reaches it, or disable the affected function), and sets an expiry date by which the exception is reviewed or the library upgraded. Choosing and validating third-party and open-source software before it is adopted is covered in the lesson on using verified secure software.

Place each test at the earliest stage that has what it needs: SAST and SCA need only code and the dependency manifest, so they run at commit or build time; DAST needs a running deployment, so it runs against staging; IAST runs wherever functional tests exercise a deployed build. Feedback is cheapest when it arrives while the developer still has the change open.

1. Commit / pull request: SAST (incremental), secret scanning, SCA of changed dependencies.
2. Build: full SAST and SCA; produce the component inventory; scan the container image.
3. Deploy to test / QA: functional and regression suites with an IAST agent attached.
4. Staging: authenticated DAST, load and failover tests, against masked data.
5. Pre-release / periodic: manual penetration testing and abuse case review for significant changes.

The order follows from what each test needs. A test placed before its prerequisites exist has nothing to examine, and a test placed later than necessary delays feedback and makes fixes more expensive. Mature pipelines therefore combine methods: early code and dependency analysis for fast feedback, and dynamic testing once a running build exists.

Tuning the gate. A gate that fails the build on every finding of any severity produces so many failures, many of them false positives, that developers learn to route around it, and a bypassed gate is worse than a looser one that is respected. The fix keeps the gate in the release path and makes its failures meaningful: narrow what can fail the build to serious findings the tool is confident about, and send everything below that line to the backlog with an owner and a deadline. Tighten the threshold over time as the backlog falls.

Abuse case (or misuse case) testing describes how an attacker or a dishonest user would deliberately misuse a function, then tests that the application resists each such attempt. A use case says what an actor should be able to do; an abuse case inverts it into what an actor must not be able to do.

The best starting point is therefore what the requirements already say each role may do: for each permitted action, ask how it could be repeated, skipped, reordered, escalated or pointed at someone else's data. What makes a test an abuse case is intent. Exercising a legitimate action at its limits is a boundary test, and measuring volume is a performance test; an abuse case asks whether a deliberately hostile or dishonest use is refused.

Examples:

• A discount-code feature: a shopper applies a single-use code in two sessions at the same moment, or edits the discount value in the request before it is sent. Tests confirm that the code is consumed once and that the server recalculates the price.
• A password-change function: a caller changes the password without supplying the current one, or changes it for a different user ID. Tests confirm that re-authentication and ownership checks are enforced.
• A cloud API: replay a captured request, call an administrative operation with a standard token, or page through other tenants' records by changing an identifier.

Abuse cases are written while requirements are being specified, so they shape both the design and the test plan, and each one should end up as an automated test in the regression suite. They are closely related to threat modelling (STRIDE, PASTA and others), which is taught in the lesson on applying the secure SDLC: threat modelling identifies what could go wrong in the design; abuse cases turn those threats into concrete, testable scenarios.

Quality assurance confirms that the organisation's defined development and test processes are actually followed, so that the software meets its requirements — including its security requirements — and it does so independently of the people who wrote the code. Testing (quality control) finds defects in a product; QA assures the process that produces and tests the product.

QA is not the team that writes the requirements, not the penetration testers, and not the risk owner who accepts residual risk on the business's behalf. Its job is to check that requirements exist, that each one is verified by a test, that the tests were run and passed, and that the release criteria are met before go-live.

Independence is the point. When the people who write the code also design and run all of its tests and approve its release, nobody independent checks coverage or decides whether the evidence is sufficient: a separation-of-duties gap. More tooling or peer review raises code quality, but the evidence is still judged by the team that produced it. The remedy for missing independence is independence: give the coverage check and the release sign-off to a quality function that reports outside the development team.

Traceability. A requirements traceability matrix links each security requirement to the test that verifies it. Passing tests prove only what they test, so a requirement with no linked test is unverified however green the pipeline dashboard looks. Where the secure SDLC makes traceability a release criterion, QA's job is to report accurately that the criterion is not met until every requirement is verified; whether the business ships anyway is a risk decision for the accountable owner, made with that accurate report in hand, not a decision QA makes by quietly relaxing the criteria.

Customers may test only what they control, within the provider's testing policy and their own rules of engagement, and they must protect production data in every test and QA environment.

Authorisation and scope

Before penetration testing an application hosted on IaaS or PaaS, check what the provider's published testing rules allow and complete whatever sign-off or advance notice they demand; policies list which services may be tested and which activities (such as denial-of-service testing) are prohibited. Then write rules of engagement covering scope, timing, contacts and stop conditions.

Scope follows the shared-responsibility line. Under IaaS the customer can test its own virtual machines, networks it configures and applications. The hypervisor and the shared management plane belong to the provider and are shared with other tenants; testing them would attack infrastructure the customer does not own and other tenants depend on, so they are out of scope and covered by the provider's attestation (independent audits and its own testing).

For SaaS, the customer usually controls almost nothing below the application's configuration, and many providers prohibit customer-initiated testing because any test touches shared infrastructure. When the terms forbid testing, assurance comes from the provider's evidence instead: its independent penetration test summary, its SOC 2 Type II report, and certifications such as ISO/IEC 27001 and 27017 or a CSA STAR entry. The time to secure testing or audit rights is contract negotiation.

Test environments and test data

Production data copied into a test or QA environment carries all its legal obligations into an environment with weaker controls and wider access. The safer substitutes are de-identified or generated records shaped like the real ones, so tests still behave realistically. The techniques differ on two questions:

When the substitution runs inside production, the test copy arrives already de-identified: no real personal value ever crosses into the weaker environment. Data protection techniques such as masking, tokenisation and anonymisation are covered in depth in the Cloud Data Security domain.