Responding to AWS Security Events: Compromised EC2, Credentials, and S3

Live response starts with validation, not action. A GuardDuty finding, a Security Hub alert, or an AWS Health notification is a claim about your environment; Task 2.2 explicitly tests your ability to validate findings from AWS security services to assess scope and impact before you contain anything. Confirm the resource exists and matters (tags tell you the owner and data classification), pull the finding's details — the API calls, the remote IP, the credential involved — and pivot into Amazon Detective or CloudTrail to see whether the flagged activity is isolated or one thread of something larger. Containing one instance while the attacker holds the instance role's credentials is theater.

Then hold the three response stages apart, because exam options love to blur them. Containment stops the spread while preserving evidence: isolate the instance, deactivate the key, block the public access. Eradication removes the threat: delete malware and backdoor principals, terminate the compromised host. Recovery restores service from known-good sources: clean AMIs, restored backups, rotated secrets. The discipline is that eradication never precedes evidence capture, and recovery never reuses artifacts you have not verified as clean.

Say GuardDuty reports Backdoor:EC2/C&CActivity.B on a production instance. The instinct is to terminate it. That instinct destroys your evidence — the instance store, the memory, often the EBS volumes if delete-on-termination is set. The correct opening moves contain the instance while keeping it intact.

Network isolation. Replace all of the instance's security groups with a dedicated isolation security group that contains no inbound rules and no outbound rules. Here is the nuance the exam loves: security groups are stateful and use connection tracking, so removing rules does not cut established, tracked connections — an attacker's live reverse shell can survive the security-group swap and linger until the connection ends or times out. (Untracked connections — flows that were permitted by broad allow rules in both directions without tracking — are interrupted immediately; the tracked ones persist.) To sever active flows deterministically, apply network ACL deny rules on the subnet: NACLs are stateless and evaluate every packet, so a deny kills established sessions at once. Use the NACL approach with care if the subnet hosts other workloads — a subnet-wide deny is a blunt instrument, which is why moving the instance's ENI behind an empty security group remains the standard per-instance isolation and the NACL is the escalation for cutting live sessions.

Structural detachment. Detach the instance from its Auto Scaling group (or move it to standby) so the ASG launches a healthy replacement and — critically — does not terminate your evidence as unhealthy; note that Auto Scaling can terminate instances regardless of API termination protection, which is exactly why detaching comes first. Deregister it from any load balancer target group so no user traffic reaches it. Then enable termination protection (DisableApiTermination) to stop an accidental or malicious terminate call.

With the instance contained, capture evidence in order of volatility — and only then destroy anything.

Snapshot before you touch. Create EBS snapshots of every attached volume before any investigation on the box, before patching, and absolutely before shutdown or termination. Snapshots are point-in-time, block-level copies; taken now, they preserve the disk exactly as the attacker left it. Every action you take on the live system after this point alters state, so the snapshot is your defensible baseline.

Capture volatile memory. If your response calls for memory forensics, capture RAM while the instance is still running — memory does not survive a stop or terminate. Conceptually this means running a memory-acquisition tool on the instance (ideally pre-staged, or pushed via Systems Manager) and writing the image to evidence storage; the Automated Forensics Orchestrator for Amazon EC2 automates exactly this capture-and-store flow. This is also why you do not shut the instance down until evidence capture is complete: stopping loses memory, and on instance-store-backed instances it loses the ephemeral disks too.

Cut its identity. The instance profile's temporary credentials may already be exfiltrated — GuardDuty's UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWS finding exists precisely for this. Revoke the role's active sessions with the aws:TokenIssueTime deny pattern so stolen credentials die even before they expire.

Tag, quarantine, and hand off. Tag the instance and snapshots as evidence under investigation, record who did what and when, and move analysis to the forensics account. Only after capture is complete do you eradicate — terminate the instance — and recover by deploying a replacement from a known-clean, patched AMI, never by reusing the compromised host or an image taken from it.

Credential incidents split on one axis: long-term versus temporary credentials.

Long-term IAM user access keys — leaked in a repository, a laptop theft, a paste site — are contained by deactivating the key immediately. Deactivation is instant and reversible, and it preserves the key ID you will pivot on in CloudTrail. AWS may beat you to partial containment: when AWS detects a publicly exposed key it raises an AWS Health event and can attach the AWSCompromisedKeyQuarantine managed policy to the user, denying high-risk actions — treat that as a signal to run your full playbook, not as remediation. Delete the key only after scoping.

Temporary credentials — a compromised role, stolen STS session tokens, exfiltrated instance-profile credentials — cannot be deactivated or recalled. Contain them with an inline deny policy on the role: "Effect": "Deny", "Action": "*", with "Condition": {"DateLessThan": {"aws:TokenIssueTime": "<now>"}}. All sessions issued before that moment lose every permission instantly; services and users regain access by fetching fresh credentials. If federation is involved, also review the role's trust policy for tampering.

Scope and hunt persistence. Query CloudTrail for everything the credential did: which APIs, from which IPs, against which resources. Then hunt for the persistence a competent attacker plants: look for CreateUser, CreateAccessKey, CreateLoginProfile, CreateRole, AttachRolePolicy, PutUserPolicy, and UpdateAssumeRolePolicy events — each is a new door that survives your key rotation. Finally, rotate every secret the credential could read (Secrets Manager makes this a controlled operation), because scoping tells you what was reachable, not what was copied.

An S3 exposure event — a bucket made public by a policy change, an overly broad ACL, or an exfiltration finding — has a short containment step and a long analysis tail.

Containment. Enable S3 Block Public Access on the bucket (or, if policy allows, confirm it at the account level, where it overrides every bucket-level setting) — it cuts off public policy- and ACL-based access in one switch without you having to reason about each statement under pressure. Then review and correct the bucket policy and ACLs themselves: remove "Principal": "*" grants, check for suspicious cross-account principals added by an attacker, and confirm object ownership and ACL posture. If credentials were involved in the exposure, fold in the credential playbook — revoke and rotate.

Analysis: what was actually read? This is where preparation decides what is knowable. CloudTrail data events record object-level API activity — GetObject, PutObject, DeleteObject — with the caller identity and source IP, but only if data-event logging was enabled on the bucket before the incident; it is not on by default. S3 server access logs are the second source: delivered on a best-effort basis and less structured, but they include authenticated and unauthenticated (anonymous) requests, which matters precisely when the bucket was public. Query either with Athena to enumerate what was retrieved, by whom, from where, during the exposure window. If neither source was enabled, you must assume everything in the bucket was exposed — an answer the exam expects you to recognize as the consequence of skipping preparation.

Recovery. Rotate any credentials or secrets that were stored in the exposed objects, notify per your obligations, and feed the root cause — how the policy went public — into preventive controls.

Analysis happens in a dedicated forensics account, not in the compromised one — the attacker's credentials have no reach there, analysis tooling cannot contaminate production, and access is restricted to the forensics team.

Moving the evidence. The standard flow: share the EBS snapshots with the forensics account by modifying the snapshot's create-volume permissions, copy them inside the forensics account (so you own an independent copy the source account cannot delete), create volumes from the copies, and attach those volumes to a hardened analysis instance as secondary, non-boot devices for examination. One sharing nuance is a reliable exam trap: snapshots encrypted with the default AWS managed key (aws/ebs) cannot be shared, because you cannot grant another account access to an AWS managed key. Evidence sharing requires the snapshot to be encrypted with a customer managed KMS key whose key policy grants the forensics account decrypt and create-grant permissions — or re-encrypting via a snapshot copy to such a key first. Prepared teams standardize customer managed keys for exactly this reason.

Chain of custody. Evidence you cannot account for is evidence you cannot rely on. Record who captured each artifact, when, from which resource, and every subsequent access; compute and store hashes of images; keep evidence in restricted, write-once-minded storage; and work on copies, never originals. Your CloudTrail record of the snapshot, share, and copy calls is itself part of the custody documentation.

Scoping and timeline. Amazon Detective is the root-cause and blast-radius tool the guide names: its behavior graph lets you pivot from a finding to the full history of the involved role, instance, and IP addresses. For precise timelines, run Athena over the CloudTrail archive — filter by access key ID, role session name, or source IP and order by event time to reconstruct the attack minute by minute.

Humans do not scale to finding volume, and the first minutes matter most. Task 2.2 expects you to know the standard automation wiring: GuardDuty findings flow to EventBridge (directly, or via Security Hub's aggregated stream), an EventBridge rule matches on the finding type and severity, and the target executes containment — a Lambda function for API-level actions or a Systems Manager Automation runbook for multi-step procedures on instances.

Map triggers to responses by finding type. A CryptoCurrency:EC2/BitcoinTool.B or Backdoor:EC2/C&CActivity.B finding can trigger the instance-quarantine runbook: swap to the isolation security group, detach from the ASG, enable termination protection, snapshot volumes, tag as under investigation. An UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.OutsideAWS finding can trigger a Lambda that revokes the role's sessions and opens an incident. The actions are the same ones you would take by hand — automation just executes the rehearsed runbook in seconds.

Security Hub custom actions cover the human-triggered middle ground: an analyst reviewing findings selects one and invokes a custom action, which emits an EventBridge event carrying the finding to the same responder targets. That gives you one containment codebase with two entry points — automatic for high-confidence findings, analyst-initiated for judgment calls.

Design rules carry over from Task 2.1: responders run with tightly scoped IAM roles of their own, reversible containment (isolate, deactivate, snapshot) can be fully automatic, and destructive or wide-impact actions sit behind an approval step. Log every automated action — your automation is itself an actor in the incident timeline you will later reconstruct.

Response is two-directional between you and AWS. When AWS notifies you: AWS Health raises events when AWS detects your access keys exposed in public repositories or observes your resources behaving maliciously; Trust & Safety sends abuse notifications to your registered security and operations contacts. These arrive only where contact information is current — and they are compromise signals that should enter the same triage pipeline as GuardDuty findings, via Health events on EventBridge. When you notify AWS: report abuse originating from AWS resources (a third party's instance attacking you) through the AWS abuse-reporting channel with logs, timestamps, and source IPs; for incidents in your own account, open a case with AWS Support — under the shared responsibility model AWS will not investigate inside your workloads for you, but Support can assist with service-side questions and constraints during response.

Post-incident activity is scored material, not ceremony. Run the root-cause analysis: not just "an instance was compromised" but the causal chain — unpatched service, permissive security group, over-broad instance role. Amazon Detective supports this backwards walk through entity behavior. Write the lessons-learned review while memory is fresh: what was detected late, which runbook steps failed or confused, how long containment took. Then update controls and playbooks: patch baselines, tighten the offending policies, add the detection that was missing, fix the runbook, and feed new scenarios into the next game day. An incident whose lessons change nothing is scheduled to repeat — and the exam's post-incident answers consistently favor the option that closes the loop over the one that only restores service.