Optimize AWS Applications: Concurrency, Caching, and Right-Sizing

Concurrency is the number of requests your application handles at the same moment — requests in flight right now — as opposed to throughput, which is requests per second. The two are linked by duration: concurrency roughly equals request rate multiplied by average duration in seconds. A function receiving 100 requests per second that takes 2 seconds each needs about 200 concurrent executions; cut the duration to 500 ms and the same traffic needs only 50. That arithmetic is why every optimization in this lesson — faster code, caching, batching — also reduces the concurrency you consume.

In Lambda, each concurrent request runs in its own execution environment, so concurrent executions equals in-flight requests. Two settings matter from the optimization angle. Reserved concurrency caps a function's maximum concurrency and guarantees it that share of the account's pool — use it to protect a downstream database from a traffic spike, not to make anything faster. Provisioned concurrency keeps a configured number of execution environments initialized ahead of traffic, so requests served by them never pay cold-start latency. When a question asks for consistently low latency on a latency-sensitive or spiky function, provisioned concurrency is the lever; when it asks to limit pressure on a downstream system, reserved concurrency is.

On containers the same concept applies per task: each ECS task handles some number of simultaneous requests. Scaling out adds concurrency capacity, while faster handlers reduce the demand for it — usually the cheaper direction to push first.

Opening a new database connection inside every request is one of the most common concurrency-related bottlenecks. Each connection costs a TCP and TLS handshake plus database authentication — often tens to hundreds of milliseconds — and a relational database supports only a finite number of simultaneous connections.

The first fix is reuse. In Lambda, initialize SDK clients and database connections outside the handler, in the initialization code that runs once per execution environment. Warm invocations then reuse the open connection instead of re-establishing it. The same applies to HTTP clients: keep-alive connections to downstream services avoid repeated handshakes on every call.

Reuse alone is not enough under high concurrency, because Lambda execution environments share nothing: 1,000 concurrent executions can mean 1,000 database connections even with perfect per-environment reuse. That is where Amazon RDS Proxy earns its place as the optimization answer for Lambda-to-RDS or Lambda-to-Aurora traffic. RDS Proxy maintains a warm pool of database connections and multiplexes many function invocations across them, so connection setup latency disappears from the request path and a burst of concurrency can no longer exhaust the database's connection limit.

In long-running compute such as ECS, use a bounded connection pool in your application framework, sized so that the number of tasks multiplied by the pool size stays safely below the database's connection limit — with headroom for deployments that briefly run old and new tasks side by side.

Profiling means measuring where a request actually spends its time — CPU, memory, and above all waiting on downstream calls — before you change anything. Optimizing without a profile is guessing, and the exam consistently rewards the answer that measures first.

Start wide with CloudWatch metrics to find which component is slow: Lambda's Duration metric, API Gateway latency metrics, and ECS CPUUtilization and MemoryUtilization narrow the search to one function or service. Then go deep with AWS X-Ray: a trace shows the full request as a timeline, and subsegments wrap each downstream call — every DynamoDB query, every outbound HTTP request — with its own start time and duration. Reading the trace waterfall exposes the hot path immediately: one slow dependency, or fifty fast calls issued sequentially that add up to seconds.

Application logs are the third source. Lambda writes a REPORT line for every invocation with Duration, Billed Duration, Max Memory Used, and — on cold starts only — Init Duration, which makes cold-start impact visible without any extra tooling. CloudWatch Logs Insights aggregates these: query for the average and the p99 of duration. The distinction matters, because an average hides tail latency; a p99 that is ten times the average is the signature of an intermittent bottleneck such as cold starts or connection setup.

Amazon CodeGuru Profiler once offered continuous CPU profiling for this job; know that it exists, but the current exam centers on X-Ray, CloudWatch metrics, and logs.

Determining the minimum memory and compute an application needs is an exam skill because on AWS, size is a dial you can turn without touching code.

Lambda has one dial: memory, from 128 MB to 10,240 MB, and CPU scales proportionally with it — at around 1,769 MB a function has the equivalent of one full vCPU. Because you pay for memory multiplied by duration, more memory is not automatically more expensive: a CPU-bound function given twice the memory may finish in less than half the time, making it both faster and cheaper. The methodology is a memory sweep: run a representative payload at several memory settings, record duration and computed cost at each, and pick the setting that minimizes whichever you care about most. AWS Lambda Power Tuning, a widely used open-source Step Functions workflow, automates exactly this sweep and visualizes the cost-versus-speed trade-off. When a question says "reduce the cost of a Lambda function without changing code", memory tuning is the intended answer.

For ECS, right-sizing means reading utilization: compare task-level CPUUtilization and MemoryUtilization against the configured task size over a peak traffic window. Sustained low utilization means the task is oversized and you are paying for idle headroom; sustained high utilization, throttling, or out-of-memory restarts mean it is undersized. Set the task size just above observed peak demand, keep headroom for spikes, and re-check after each significant release — right-sizing is a recurring activity, not a one-time event.

Subscription filter policies move message filtering out of your code and into the messaging service, so each subscriber receives only the messages it actually needs. Without one, an SNS topic delivers every message to every subscriber, and each consumer burns invocations, compute, and downstream requests discarding messages it was always going to ignore — filtering after delivery is pure waste.

An SNS filter policy is a JSON document attached to a subscription. By default it matches against message attributes; with the filter policy scope set to the message body, it matches against the payload itself, which helps when publishers do not set attributes. Publishers change nothing either way — the topic evaluates the policy and skips non-matching deliveries. The exam trigger is unmistakable: "subscribers should receive only relevant messages without modifying publisher or consumer code" means an SNS subscription filter policy, not an if-statement in the consumer.

Amazon EventBridge applies the same principle at the event bus. A rule's event pattern filters on event content — exact values, prefixes, numeric ranges, existence checks — and only matching events reach the rule's targets. Tightening an over-broad event pattern is a legitimate optimization in its own right: fewer target invocations, less downstream noise, no code change.

Both mechanisms share one caution: a message that matches no subscription filter or rule is simply not delivered. Filter deliberately, and test policies against real message shapes before relying on them in production.

SQS charges per request, not per message, which makes batching the most direct optimization on both sides of a queue. On the producer side, SendMessageBatch packs up to 10 messages into a single call — one network round trip and one billed request instead of ten. On the consumer side, ReceiveMessage returns up to 10 messages per call when you raise MaxNumberOfMessages, and DeleteMessageBatch acknowledges them in one call. For a Lambda consumer, the event source mapping already delivers batches; sizing the batch appropriately amortizes per-invocation overhead across more messages.

Polling style is the second half. With short polling, ReceiveMessage samples a subset of SQS servers and returns immediately — often empty — so an idle consumer generates a steady stream of billed, useless requests. With long polling, the call waits up to WaitTimeSeconds (maximum 20 seconds) for a message, returning as soon as one arrives. That eliminates most empty responses and reduces delivery latency, because the consumer is already waiting when the message lands. Enable it per queue with ReceiveMessageWaitTimeSeconds or per call with WaitTimeSeconds.

"The application receives many empty responses from SQS" or "reduce polling cost on a mostly idle queue" both point straight at long polling.

Caching content based on request headers starts with the cache key — the set of request attributes that decides whether two requests can share a cached response. In CloudFront, the cache policy defines exactly which headers, query strings, and cookies join the cache key. If a response varies by Accept-Language, add that header to the cache policy and CloudFront stores one copy per language. The classic anti-pattern is including or forwarding all headers: nearly every request then has a unique key, the hit ratio collapses, and your "cache" forwards everything to the origin. Include only the attributes that genuinely change the response.

API Gateway caches per stage: enable the stage cache and responses are served from it for the TTL — 300 seconds by default, 3,600 at most. Method-level settings vary caching per resource, and cache keys can incorporate request parameters so that, for example, a query-string parameter produces separately cached entries. Origins steer every downstream cache with Cache-Control headers: max-age for how long a response may be reused, no-store for never.

Application-level caching is the layer the v2.1 guide added. A warm Lambda execution environment persists between invocations, so data fetched into a module-scope variable — configuration, reference data — is free on every warm invocation; it is per-environment, so treat it as best-effort and revalidate on a timer. For a cache shared across all environments and services, ElastiCache with the cache-aside pattern applies: check the cache, and on a miss read the database and populate the cache with a TTL. Cache what is read often and changes rarely, and give every entry an explicit TTL — unbounded lifetimes turn stale data into a correctness bug.

Performance analysis is a loop, not a bag of tricks: measure, find the constraint, fix one thing, re-measure. Fixing anything other than the current constraint changes nothing a user can see, and fixing two things at once means you cannot tell which one worked.

A few bottleneck patterns account for most exam scenarios. Chatty calls — many small sequential requests to the same service — are fixed by batching (BatchGetItem, SendMessageBatch) or by parallelizing independent calls. The N+1 query, one lookup per item in a list, is the same disease with a database accent. Synchronous chains, where a request waits on work it does not need before responding, are fixed by handing that work to a queue or event bus and replying immediately. Oversized payloads are fixed with compression and pagination.

Walk the loop on a real symptom: an API's p99 latency is 4 seconds while its average is 300 ms. Logs Insights confirms the tail is real and not cold starts — Init Duration appears in too few REPORT lines to explain it. An X-Ray trace of a slow request shows the handler issuing a DynamoDB GetItem per order line: 40 sequential subsegments of a few milliseconds each. First fix: replace the loop with BatchGetItem. Re-measure — p99 drops to about 900 ms. The trace now shows a repeated read of the same reference data on every request; second fix: cache it, in the warm environment or ElastiCache, with a sensible TTL. Re-measure — p99 lands near 350 ms. Two targeted changes, each one verified, and no re-architecture.