Auto Scaling and Elasticity: EC2, Caching, and Database Scaling

An EC2 Auto Scaling group (ASG) keeps a fleet of instances between three numbers: min, max and desired capacity. Scaling policies move desired up and down within the min/max bounds, and the group launches or terminates instances until the running fleet matches it. Note what each bound does: min is a floor that blocks scale-in below it, and max is a ceiling that blocks scale-out above it. A group whose desired capacity equals its minimum can still scale out freely.

What gets launched is defined by a launch template (AMI, instance type, security groups, key pair, user data). Templates are versioned, so you roll a new AMI forward by pointing the group at a new version and roll back just as easily. Launch templates replaced the older launch configurations, which cannot be edited and no longer receive new features. A mixed instances policy lets one group launch several instance types (and Spot as well as On-Demand), which matters when a single type runs short of capacity. Spot allocation strategies such as capacity-optimized can only choose among the pools you allow, so they cannot help a group that lists one instance type in one Availability Zone.

The group also replaces unhealthy instances. By default it uses EC2 status checks: an instance that fails them is terminated and replaced. Behind a load balancer, add the ELB health check type so an instance whose application is broken (but whose VM is fine) is replaced too. The health-check grace period is the time after launch before the group starts acting on a new instance's health checks. Set it shorter than the application's real startup time and the group terminates instances that were still booting, then launches replacements that suffer the same fate.

So for slow-starting instances, lengthen the grace period rather than loosening the target group's health check. AZ rebalancing keeps instances spread evenly across the group's Availability Zones, launching in an underweighted zone before terminating from an overweighted one. Which instances leave during scale-in is covered in its own section below.

Five policy types adjust desired capacity, and each answers a different kind of requirement.

Target tracking is the modern default. Policies can coexist on one group. When several dynamic or predictive policies evaluate at once, the group follows the one calling for the most capacity. A scheduled action is different: it sets capacity values once, so pair it with target tracking by having it raise the minimum (see below).

Scheduled actions: set the floor, not just desired

A scheduled action that only raises desired capacity is overruled as soon as a dynamic policy sees the metric below target: target tracking scales the group back in, sometimes before the peak arrives. To pre-warm for a known event, have the scheduled action raise min (the floor dynamic policies respect), then add a second scheduled action that lowers min again after the peak so normal scale-in can resume.

Predictive scaling in practice

Predictive scaling learns from the group's own load history (it needs at least 24 hours of data to start forecasting and works best on clear daily or weekly cycles) and creates capacity before the forecast load arrives. It has two modes: forecast only, which produces forecasts you can compare with actual load without changing capacity, and forecast and scale, which acts on them. Starting in forecast only mode, alongside the existing dynamic policy, is how you gather evidence before trusting it. A scheduling buffer tells it to launch instances that many seconds before the forecast need, so instances with long boot times are ready in time. Compared with scheduled actions, predictive scaling follows a peak whose time or size drifts from week to week, while a scheduled action fires at the clock time you wrote, whatever the load does. Keep a target tracking policy alongside for load the forecast did not predict.

Target tracking only works when the metric falls as capacity is added and rises as capacity is removed. Average CPU does that for CPU-bound fleets, but other workloads need a different metric.

Backlog per instance is the AWS-recommended metric for queue workers. Build it with metric math: the queue's ApproximateNumberOfMessagesVisible divided by the group's GroupInServiceInstances. The target is a number of messages, not seconds: the acceptable wait time divided by the average processing time per message. For example, if a message may wait at most 120 seconds and each takes 4 seconds to process, one instance can work through a backlog of 30 messages in time, so the target is 30. Target tracking then adds workers whenever each instance has more than 30 messages waiting. Setting the target to the wait time itself confuses seconds with messages.

Cooldowns and warmup exist because a freshly launched instance contributes nothing for its first minutes, and scaling decisions made during that window overreact. The two settings apply to different policy types.

This is why changing the default cooldown has no effect on a target tracking group that overshoots: the setting that matters there is instance warmup. Set warmup to roughly the time an instance takes to boot and start serving. During warmup, the group neither scales out again for load the new instance is about to absorb nor treats the booting instance's near-zero CPU as a reason to scale in. Step scaling also keeps responding while an activity is in progress (a bigger breach can trigger a further step), whereas simple scaling cannot act again until its activity and cooldown are both over. Warmup is a different knob from the health-check grace period: grace delays health evaluation; warmup delays metric participation.

Lifecycle hooks pause an instance mid-transition. A launch hook holds a new instance in Pending:Wait so you can install software, register it with tooling or pre-warm a local cache before it goes into service; a termination hook holds it in Terminating:Wait so you can drain work and ship logs off the box. If a group launches instances that hang before entering service, an uncompleted launch hook is a prime suspect. The termination side, and how long a hook can hold an instance, is covered next.

When a group scales in, three controls decide what happens, and they act at different points: scale-in protection decides whether an instance can be chosen, the termination policy decides which eligible instance is chosen, and a termination lifecycle hook decides how long the chosen instance waits before it terminates.

Scale-in protection does not stop the group replacing an instance that fails its health checks, and it does not stop you terminating the instance yourself. A hook never saves a selected instance: completing it with CONTINUE or ABANDON, or letting it time out, all end with a terminating instance terminated (ABANDON also skips any remaining hooks).

Termination policies

The default policy first balances instances across Availability Zones, then (for mixed instances groups) follows the allocation strategy, then prefers instances on the oldest launch template or configuration, then the instance closest to its next billing hour. You can set others in order:

• OldestLaunchTemplate: removes instances whose launch template or version is not the current one first, even if those instances were launched recently. This is the one to choose when the goal is to retire instances still running a previous template version.
• OldestInstance / NewestInstance: by launch time, regardless of template. Oldest helps cycle out long-running instances; newest suits testing a new configuration you may want to remove.
• ClosestToNextInstanceHour: the instance nearest its next billing hour.
• AllocationStrategy, and a custom policy backed by a Lambda function, for finer control.

Availability Zone balance always comes first: a policy chooses among instances in the zone the group is reducing.

How long a lifecycle hook can hold an instance

A hook's heartbeat timeout is 30 to 7,200 seconds (default 3,600). When the timeout passes without a response, the hook's default result is applied and the transition continues. For work that takes longer, the instance calls RecordLifecycleActionHeartbeat to restart the timeout, up to a maximum of 48 hours or 100 times the heartbeat timeout, whichever is smaller. When the work ends, it calls CompleteLifecycleAction so the instance does not sit idle for the rest of the timeout. Jobs that can run for longer than that maximum need scale-in protection instead.

Walk through the classic flapping scenario. A group runs a step policy scaling out when average CPU exceeds 70% and a second policy scaling in below 30%. Load rises and two instances launch. While they boot, their near-idle CPUs drag the group average under 30%, so the scale-in policy terminates instances. Load concentrates on fewer instances, CPU spikes past 70%, and the cycle repeats every few minutes. The fixes: replace both policies with one target tracking policy, which handles scale-out and scale-in together and scales in conservatively; set instance warmup so booting instances stop distorting the average; or, if step policies must stay, widen the gap between the out and in thresholds. Changing the cooldown does not fix it: the cooldown does not apply to step or target tracking policies, and the feedback loop remains.

Alarm in breach, nothing launching. Start with the group's activity history and compare desired with max. The causes fall into three groups:

Suspended processes. You can suspend individual Auto Scaling processes, usually for maintenance, and forgetting to resume them is a common cause of a group that ignores load. The group can also suspend processes itself after repeated launch failures. Know what each one stops:

• AlarmNotification: the group ignores the CloudWatch alarms behind its scaling policies, so target tracking and step policies stop changing desired capacity.
• ScheduledActions: only scheduled actions stop; alarm-driven policies still work.
• Launch: the group cannot launch instances, whether for scale-out or replacement.
• Terminate: the group cannot terminate instances, so scale-in stops.
• HealthCheck: the group stops marking instances unhealthy. ReplaceUnhealthy: unhealthy instances are not terminated and replaced.
• AZRebalance: the group stops rebalancing across Availability Zones. AddToLoadBalancer, InstanceRefresh: as named.

The same scaling vocabulary carries to containers, but ECS has two scaling layers and both must work.

On Fargate there are no instances to manage, so service auto scaling is the whole answer. On EC2-backed clusters, raising the task count only helps if there is room to place tasks; when service events report that no container instance has enough CPU or memory, tasks are waiting for instances. Create an Auto Scaling group capacity provider with managed scaling and update the service to use a capacity provider strategy that includes it (a service that uses a launch type instead of a strategy does not drive managed scaling). ECS then grows the group to fit the tasks waiting to be placed.

Lambda scales the number of concurrent executions automatically with incoming requests, so there is no scaling policy to write. Its controls are about concurrency:

Reserved concurrency does not make environments warm, and provisioned concurrency is the only one that removes initialization delay. When invocations are throttled, the function has reached its reserved cap or the account quota; the fix is a concurrency change, not an Auto Scaling group.

Caching is a scaling mechanism: every request served from a cache is a request your fleet and database never see. CloudFront serves static assets and cacheable dynamic responses (article pages, product pages, API reads) from edge locations, so a higher cache hit ratio shrinks the fleet the Auto Scaling group must keep. For a web tier overloaded by repeated requests for the same content, this is usually cheaper than adding instances; an ElastiCache cluster behind the application relieves the database, not the web tier.

The hit ratio depends on the cache key, which a cache policy defines: the URL plus whichever query strings, headers and cookies you include. Every value you include splits the cache into more variants, so a cache key that includes all query strings and all cookies turns every value that does not affect the response (tracking parameters, session or analytics cookies) into a separate cache entry and drives the hit ratio toward zero. Include only what changes the response. If the origin still needs a value that is not in the cache key, forward it with an origin request policy, which sends values to the origin without adding them to the cache key. TTLs decide how long an object is served before CloudFront checks the origin again; short TTLs mean more origin requests.

CloudFront caches in two layers: edge locations, and regional edge caches behind them. Origin Shield adds a third, central layer in one Region you choose (normally the one closest to your origin): every regional edge cache goes through it, so when many locations ask for the same object at once, such as a popular object whose TTL has just expired, the origin sees one request instead of one per regional cache.

ElastiCache sits between the application and the database. It offers three engines plus a serverless option:

Valkey is the open-source, Redis-compatible engine that ElastiCache now leads with; Redis OSS remains supported. Topology, metrics and scaling for node-based caches follow. For DynamoDB specifically, DAX plays the read-caching role (see the DynamoDB section).

A node-based Valkey or Redis OSS cache is built from shards, and the exam tests what each part scales.

With cluster mode disabled the cache has a single shard, so writes and memory are limited to one primary node. Multi-AZ with automatic failover requires at least one replica: a cache, or a cluster mode enabled shard, with no replicas has nothing to promote, so a primary failure loses that node's data until it is replaced. Node-based caches with cluster mode enabled also support ElastiCache auto scaling, which adds or removes shards or replicas through target tracking or scheduled policies.

Reading the metrics:

• CacheHitRate: the share of reads served from the cache. A falling hit rate sends load back to the database.
• Evictions: keys removed to make room because memory is full. Rising evictions mean the cache is too small for its working set: add memory (larger nodes or more shards), not replicas.
• Reclaimed: keys removed because their TTL expired. A hit rate can fall because TTLs are short while Evictions stays flat; that is a TTL decision, not a capacity problem.
• DatabaseMemoryUsagePercentage: how full the memory is; together with Evictions, the signal to add capacity.
• EngineCPUUtilization versus CPUUtilization: Valkey and Redis OSS run commands on one main thread, so on a multi-vCPU node the host-wide CPUUtilization can look low while the engine thread is saturated. Alarm on EngineCPUUtilization to catch it.

RDS scales vertically by modifying the DB instance class. On a Single-AZ instance the change restarts the database, so it causes an outage, either immediately or in the maintenance window. With a Multi-AZ deployment, RDS applies the change to the standby first, fails over to it, then updates the old primary, so the interruption is about the length of a failover (typically one to two minutes) instead of a full resize and restart. The application keeps the same endpoint throughout, because failover repoints the instance's DNS name.

Converting a Single-AZ instance to Multi-AZ needs no downtime: RDS takes a snapshot, builds the standby from it in another Availability Zone and starts synchronous replication, though there can be extra latency while it does so. That makes "convert to Multi-AZ, then resize" a way to change the class with only a brief failover interruption. Compare promoting a read replica: the promoted replica becomes a separate standalone instance with its own endpoint, so applications must be repointed.

Storage autoscaling handles the other vertical axis: when free storage runs low, RDS grows the volume automatically up to a maximum you set. It only grows; storage never shrinks.

For read-heavy load, add read replicas. Replication is asynchronous, so replicas serve eventually consistent reads and the application must send read traffic to them. Alarm on ReplicaLag: rising lag means stale reads now and a longer catch-up if you ever promote the replica. Replicas add read capacity only; they do nothing for writes.

Aurora separates compute from a shared cluster volume that grows automatically. A cluster has one writer and up to 15 Aurora Replicas, all reading the same storage with typically low lag. Aurora Replica auto scaling adds and removes replicas with a target tracking policy on average replica CPU or average connections. It helps reads only, and only the clients that reach the new replicas. Which endpoint the application uses decides that:

Traffic sent to individual instance endpoints or a static member list never reaches replicas added later, so auto scaling adds capacity nobody uses; the reader endpoint (or an exclusion-list custom endpoint) spreads reads across new replicas automatically. Aurora Serverless v2 scales an instance's capacity continuously in ACUs (Aurora capacity units) between a minimum and maximum you set, which suits spiky or unpredictable database load that a fixed instance class handles badly.

DynamoDB scales through its two capacity modes.

Throttling tells you the mode or its limits are wrong. Requests above provisioned throughput fail with ProvisionedThroughputExceededException, visible in the ThrottledRequests and read/write throttle-event metrics. A steady workload that throttles needs more capacity (or a higher auto scaling maximum). A bursty workload that throttles between auto scaling adjustments needs on-demand mode: raising the auto scaling maximum or shortening its cooldowns does not make it react faster, because it still waits for the metrics to show the load. Throttling while the table has plenty of unused capacity points to a hot partition: one partition key taking a disproportionate share of traffic. Adaptive capacity softens this automatically, but the durable fix is a higher-cardinality partition key.

For hot reads, front the table with DynamoDB Accelerator (DAX), an in-memory cache that is API-compatible with DynamoDB. The application swaps in the DAX client and keeps making the same calls, and gets microsecond reads for eventually consistent requests. ElastiCache can also cache DynamoDB data, but then the application must contain the caching logic (check the cache, read the table on a miss, write the result back, invalidate on change).