Systems Manager and event-driven automation: managing resources for SOA-C03

Systems Manager can only operate on instances it can see, and visibility rests on three prerequisites: a running SSM Agent, credentials the agent can use, and a network path to the Systems Manager endpoints. Miss any one and the instance never becomes a managed node, so Run Command, Session Manager, Patch Manager and State Manager all ignore it.

1. The agent

The SSM Agent comes preinstalled on many AWS-provided AMIs, but a custom, hardened or minimal image may lack it or carry an old version, and a stopped or crashed agent makes the instance invisible. Some features also need a minimum agent version, so an old agent baked into an in-house image is a common hidden cause.

2. Credentials: instance profile or Default Host Management Configuration

There are two ways to give the agent permissions:

• An IAM instance profile on each instance whose role includes the managed policy AmazonSSMManagedInstanceCore. Attaching a role after launch works; the agent picks up the new credentials without a relaunch.
• Default Host Management Configuration (DHMC), an account-and-Region setting that lets Systems Manager manage EC2 instances without any instance profile. You choose one IAM role (by default AWSSystemsManagerDefaultEC2InstanceManagementRole) once for the account and Region, and Systems Manager uses it on the instances' behalf — it is never attached to instances as an instance profile. Instances qualify only if they use IMDSv2 and run SSM Agent 3.2.582.0 or later; an instance with an older agent stays unmanaged even though the setting is on.

3. The network path

The agent only makes outbound HTTPS connections. It needs internet egress (internet gateway or NAT gateway) or, in a subnet with no internet access, interface VPC endpoints. Two details decide whether endpoints actually work: private DNS names must be enabled on them so the standard service host names resolve to the endpoint, and the endpoint's security group must allow inbound HTTPS (443) from the nodes.

The diagnostic rule: when an instance is missing from Systems Manager, check the agent (installed, running, recent enough), the credentials (instance profile policy, or DHMC prerequisites) and the path (NAT or the three core endpoints with private DNS and a security group that admits 443 from the nodes).

Session Manager gives you an interactive shell on a managed node without SSH keys, bastion hosts, or any open inbound ports. The SSM Agent keeps an outbound connection to the service and sessions are brokered over it, so security groups can deny all inbound traffic. Access is granted by IAM policies (for example, ssm:StartSession limited to instances with a given tag) rather than by distributed keys.

What gets recorded, and what does not

• Session logging to CloudWatch Logs or S3 captures the commands typed and their output in standard shell sessions. It is not available for sessions that connect through SSH (AWS-StartSSHSession) or port forwarding, because that traffic is opaque to the service.
• CloudTrail records the StartSession (and TerminateSession) API calls — who opened a session to which instance and when — but never the commands run inside it.
• Run As support (Linux and macOS) sets which OS user a session runs as instead of the default ssm-user, for example from a tag on the IAM principal. It controls permissions on the host; it records nothing.
• A node in an isolated subnet can only deliver logs if it can reach the destination: a logs interface endpoint for CloudWatch Logs, an S3 gateway endpoint for S3, and a kms endpoint if sessions are KMS-encrypted.

Port forwarding

AWS-StartPortForwardingSession forwards a local port to a port on the managed instance itself. AWS-StartPortForwardingSessionToRemoteHost uses the managed instance as a hop to reach another host it can connect to, such as a private RDS endpoint, so an engineer can run a database client locally with no bastion and no inbound rules.

Session Manager compared with the alternatives

Run Command runs one Systems Manager document — AWS-RunShellScript on Linux, AWS-RunPowerShellScript on Windows, or any other Command document — across one instance or thousands, once. Target by tags or resource groups rather than instance lists, so newly tagged instances are included without editing anything. It is the ad hoc, imperative tool: restart a service everywhere, collect a file, apply an urgent fix.

Rate controls: two different brakes

A small concurrency with a low error threshold is how you roll a risky command out gradually: a bad command fails on a few instances and stops there.

Output and audit

The console shows only the first 2,500 characters of each instance's output. For full output, send it to an S3 bucket or CloudWatch Logs when you run the command. CloudTrail records the SendCommand API call (who ran which document against which targets) but not the output. If the same command must run repeatedly to keep instances in a desired state, that is State Manager's job, covered below.

Automation runbooks are Systems Manager's multi-step workflows: a runbook defines a sequence of steps that call AWS APIs, run scripts, invoke Run Command on instances, branch, or wait for conditions, with each step's output available to later steps. They turn a wiki page of manual steps ('snapshot the volume, stop the instance, resize it, start it, verify') into one repeatable operation.

Predefined runbooks first

AWS ships hundreds of runbooks. Operational examples: AWS-RestartEC2Instance, AWS-StopEC2Instance, AWS-StartEC2Instance, AWS-CreateImage, AWS-UpdateSSMAgent. The AWSSupport-* runbooks handle common repairs:

• AWSSupport-ResetAccess recovers access when the SSH key or Windows Administrator password is lost: it generates a new key pair (or resets the password) using EC2Rescue on an offline volume and stores the new secret in Parameter Store.
• AWSSupport-TroubleshootSSH and AWSSupport-ExecuteEC2Rescue diagnose and repair SSH configuration and connectivity problems; they do not issue new credentials.

Step actions for custom logic

Who the runbook runs as

Without an AutomationAssumeRole parameter, a runbook runs with the permissions of the person or service that started it. To let operators run a runbook that does more than they themselves can, use delegated execution: create a role that trusts ssm.amazonaws.com and holds the permissions the steps need, pass it as AutomationAssumeRole, and give the operator only ssm:StartAutomationExecution plus iam:PassRole on that role. Runbooks can also run across many resources with concurrency and error thresholds, and across accounts and Regions from a central account.

Runbooks are the standard target of other automation: a maintenance window or an EventBridge schedule can run one at set times, and an EventBridge rule can start one in response to an event — which is how detection becomes remediation without a human in the chain.

Patch Manager is the exam's patching answer: a patch baseline defines which patches are approved, a maintenance window (or a patch policy) decides when they install, AWS-RunPatchBaseline scans or installs, and patch compliance proves the result. Walk a campaign for 200 Linux instances across three environments.

1. The patch baseline: what 'patched' means

A baseline holds approval rules (classifications such as Security, severities such as Critical) plus explicit lists:

• Approved patches: specific patches approved regardless of the rules.
• Rejected patches, with a rejected-patches action: Allow as dependency (the default) still installs a rejected package if another package needs it and reports it as InstalledRejected; Block never installs it, even as a dependency.

Each approval rule approves patches either after an auto-approval delay (a number of days after release, counted from the day the patching runs) or up to a fixed approval cutoff date. The difference matters for staged rollouts: with a delay, a dev run on Monday and a prod run a week later approve different sets, because more patches have aged past the delay by then; with a cutoff date, every environment installs the identical set. AWS provides default baselines per operating system; custom baselines encode your policy.

A baseline has no rate controls. Concurrency and error thresholds are set where the patching is run: on the maintenance window task, the Run Command invocation, or the State Manager association.

2. Which instances and when

There are two ways to wire this up. The traditional one maps instances to baselines through patch groups (the Patch Group or PatchGroup tag) and registers AWS-RunPatchBaseline as a maintenance window task. The newer, recommended one is a patch policy created in Quick Setup, which sets baselines, scan and install schedules, and targets across accounts and Regions of an organization in one configuration.

A maintenance window has a schedule, a duration (hours the window stays open) and a cutoff: the number of hours before the window ends when Systems Manager stops starting new tasks. Tasks already running continue. For a 4-hour window with a 1-hour cutoff, no new task starts in the final hour. Registered tasks have a priority: lower numbers run first, and tasks with the same priority run in parallel.

3. Scan, install, reboot

AWS-RunPatchBaseline has two operations: Scan reports missing patches and changes nothing; Install applies approved patches. A common pattern is scanning daily and installing inside the window. The RebootOption parameter controls restarts:

4. Compliance reporting

Patch Manager records per-node compliance. States to recognize: Missing (approved but not installed), Installed, InstalledPendingReboot, InstalledOther (installed on the node but not approved by the baseline — for example, added by hand), InstalledRejected and Failed. A node with any Missing, Failed, InstalledPendingReboot or InstalledRejected patch is non-compliant; NotApplicable (not relevant to the node) does not count against it.

The pattern to lock in: the baseline defines what, patch groups or a patch policy define which instances, the window or policy schedule defines when, Scan or Install defines how, and compliance provides the proof.

State Manager is the desired-state half of Systems Manager: an association binds a document to a set of targets on a schedule, and Systems Manager reapplies it continuously — correcting configuration drift rather than merely detecting it. Keep the CloudWatch agent installed on every instance with a tag, keep the SSM Agent current with AWS-UpdateSSMAgent, keep a configuration applied to every new instance that matches the target. A new association runs immediately when it is created and then on its schedule; turn on apply only at cron interval (ApplyOnlyAtCronInterval) when it must first run at the next scheduled time, for example to avoid restarting services in business hours.

Inventory collects what is on the fleet: installed applications and versions, agent versions, OS details, network configuration. Compliance reports association and patch status. To see this across accounts and Regions, a resource data sync writes the inventory of many accounts and Regions to one S3 bucket, where you query it with Athena (or visualise it in QuickSight). For a history of software changes, AWS Config can record the AWS::SSM::ManagedInstanceInventory resource type, giving a timeline of what was installed or removed on each node.

Parameter Store

Parameter Store holds operational configuration in a hierarchical, versioned store: AMI IDs, endpoint URLs, feature flags, and — as SecureString parameters encrypted with KMS — values that must not sit in plain text.

• Standard tier: no extra charge, values up to 4 KB, no parameter policies. Advanced tier: charged per parameter, values up to 8 KB, and parameter policies: Expiration (delete at a date), ExpirationNotification (EventBridge event before expiry) and NoChangeNotification (EventBridge event if not changed for a period). SecureString is available in both tiers.
• Secrets Manager costs more per secret but adds built-in rotation, cross-account sharing with resource policies, and replication across Regions; choose it for credentials that must rotate, and Parameter Store for configuration and secrets that do not.

When a parameter reference is resolved

Choosing the right Systems Manager capability is its own exam skill — scenarios are engineered so exactly one fits. The discriminators are cadence (once versus continuous), shape (single command versus multi-step workflow), and domain (general operations versus patching).

Two boundary reminders keep answers clean. Session Manager is interactive access, not automation — it never competes in these questions. And a maintenance window is less a capability of its own than the scheduler that coordinates disruptive work: it can run Run Command tasks and Automation runbooks alike inside an agreed operational window, which is why patching and other risky jobs anchor to it. If a scenario stresses that work must only happen during an approved window, the maintenance window is part of the answer regardless of which capability does the work.

Event-driven automation reacts to what just happened instead of polling for it, and EventBridge is the router. AWS services publish events to the default event bus; a rule matches an event pattern and delivers matches to targets: Lambda functions, Systems Manager Automation runbooks, Run Command, SQS queues, SNS topics, Step Functions state machines, other event buses, and API destinations. Typical wiring: an EC2 instance changes state → a runbook tags or remediates it; a Health event lands → notify the on-call topic; a Config compliance change → remediation; a GuardDuty finding → a runbook that isolates the instance.

Details that decide whether it works

• Rules are Regional. A rule sees only the AWS service events of its own Region, so cover every Region by deploying the rule in each (CloudFormation StackSets does this well) or by forwarding events to a central bus.
• Input transformers extract fields from the event (for example $.detail.instance-id) and build the target's input, which is how an event's instance ID becomes a runbook parameter.
• API destinations call a third-party HTTPS endpoint directly from a rule. A connection stores the authorization (API key, OAuth client credentials or basic auth, kept in Secrets Manager for you); you can set an invocation rate limit, and failed calls are retried. With an input transformer to shape the body, this replaces a Lambda function written only to call a webhook.
• Security Hub CSPM sends every finding to EventBridge automatically (Findings - Imported) and also supports custom actions, which an analyst triggers manually on selected findings (Findings - Custom Action) — use them for human-chosen responses, not automatic remediation.

Scheduled automation

An EventBridge Scheduler schedule always needs an execution role that Scheduler assumes to call the target. A rule needs an IAM role for targets such as Systems Manager Automation or Run Command, Step Functions and API destinations; Lambda, SNS and SQS targets of a rule are authorised by the target's resource-based policy instead. To start a runbook that uses an AutomationAssumeRole, that role needs ssm:StartAutomationExecution and iam:PassRole on the automation role.

S3 Event Notifications are the bucket-level way to react to object changes. You choose event types — s3:ObjectCreated:*, s3:ObjectRemoved:*, s3:ObjectRestore:* (Glacier restores), s3:LifecycleExpiration:*, replication and others — optional prefix and suffix filters, and a destination: a Lambda function, an SQS standard queue, or an SNS standard topic. Step Functions, FIFO queues and other services are not direct destinations; reach them through EventBridge.

Limits and validation

• S3 rejects overlapping configurations: two notifications for the same event type whose prefix/suffix filters could both match the same key (for example one with prefix logs/ and another with no filter) cannot coexist. To send the same events to several consumers, fan out through one SNS topic, or use EventBridge.
• S3 validates the destination when you save the configuration. A Lambda function needs a resource-based permission allowing s3.amazonaws.com to invoke it (in CloudFormation, add an AWS::Lambda::Permission yourself; it is not created for you). An SQS queue or SNS topic needs a policy allowing S3 to send, and if it uses SSE-KMS it must use a customer managed key whose key policy lets S3 use it — the AWS managed key cannot be granted to S3.

Turning on EventBridge for the bucket

One bucket setting sends all of the bucket's events to EventBridge (nothing arrives until it is turned on). Rules then filter on content — key prefix and suffix, object size, requester, source IP — with no overlap limit and fan out to any EventBridge target, including Step Functions. The events carry fields such as bucket, key, size, ETag, version ID, requester and source IP, but not user-defined object metadata or tags; a consumer needing those must read the object. CloudTrail S3 data events with an EventBridge rule can also react to object API calls, but they cost more and add delay compared with native notifications.

Lambda is the glue for logic AWS does not ship: parse the event, apply rules, call APIs. Step Functions orchestrates when the flow has several steps, branching, retries, parallel work or waits that one function should not hold open. The workflow type matters:

The wait-for-task-token pattern handles human approval inside a workflow: a task sends a token (for example in an email or chat message) and the execution pauses until something calls SendTaskSuccess or SendTaskFailure with that token. Inside Systems Manager, the equivalent gate is the runbook aws:approve step.

AWS DevOps Agent (generally available since March 2026) is an AWS-managed AI agent that investigates operational incidents. The SOA-C03 guide names it next to Lambda, S3 Event Notifications and EventBridge for event-driven automation. When an alert arrives — such as a CloudWatch alarm or an incident from a tool like PagerDuty or ServiceNow — it starts an investigation, correlates metrics, logs, traces, code changes and recent deployments, and returns a probable root cause and recommended mitigation steps. It also suggests improvements to help prevent repeat incidents.

By default it investigates and recommends; people make the change. Out of the box the agent has read-only access: it does not change resources, roll back deployments or run runbooks on its own initiative. Engineers review its findings and carry out the mitigation, for example by running an Automation runbook.

Since August 2026 an administrator can opt in to directed actions: operations an operator explicitly asks the agent to perform, such as executing a mitigation plan from an investigation. Actions that create or modify resources are disabled by default; enabling them is an agent-space setting backed by the AIDevOpsAgentActionsPolicy managed policy, and each action is approved by an operator and attributed to that person in CloudTrail. So under a policy that forbids automated changes to production, leave write actions off (or keep a person approving each one): the agent never decides to change resources by itself.

End-to-end automation chains these pieces, and the exam rewards recognizing whole patterns. Scheduled stop/start for cost: an EventBridge Scheduler schedule in the team's time zone invokes AWS-StopEC2Instance against tagged development instances every evening and AWS-StartEC2Instance every morning — no custom code, because predefined runbooks already do the work. A patching campaign is a baseline, patch groups or a patch policy, a maintenance window, and compliance reporting, as walked through above. An AMI refresh chain keeps golden images current: a scheduled rebuild produces a new AMI, the new AMI ID lands in a Parameter Store parameter, and launch templates that reference it with resolve:ssm: use it at the next launch — the running fleet then picks it up through its normal rollout mechanism, such as an instance refresh.

When several answers would technically work, apply the selection rule the exam repeats across services: prefer managed and declarative over custom code. If a predefined runbook, a State Manager association, Patch Manager or an EventBridge API destination covers the requirement, choosing Lambda means writing, patching, and monitoring code for an already-solved problem — the phrase 'least operational overhead' points at the managed option. Choose Lambda when the logic is genuinely custom: parse this event, apply business rules, call several APIs in a particular order. Choose a runbook when the procedure is multi-step and operational, especially when it needs an approval gate. Choose State Manager when the requirement is a state to maintain rather than an action to perform. Read the verbs in the question — run once, keep enforced, on a schedule, when X happens — and each maps to a different tool.