DVA-C03 Optimization: Lambda Concurrency, Memory, Messaging, Caching and Bottlenecks

Lambda concurrency is the number of requests a function is handling at the same moment, and each one needs its own execution environment. Two separate limits decide whether a request gets an environment: the account concurrency quota for the Region (1,000 by default, shared by every function, and raisable through Service Quotas) and the per-function scaling rate, which lets each function add up to 1,000 execution environments every 10 seconds. A request that exceeds either limit is throttled: a synchronous caller such as API Gateway receives a 429 TooManyRequestsException.

That distinction explains a common surprise. A raised account quota does not help a function that must go from idle to several thousand concurrent requests in a few seconds, because the function cannot create environments faster than its scaling rate. When such a burst happens at a predictable time, have the environments ready before it starts by scheduling provisioned concurrency.

Capping the noisy function with reserved concurrency protects the API today, but a newly added function can still consume what is left; reserving concurrency on the function that must be protected is the guarantee. Lambda always keeps at least 100 units of unreserved concurrency for functions without a reservation. Reserved concurrency does nothing for cold starts. The basic Lambda configuration settings themselves are covered in the Lambda development lesson for Task 1.2.

Provisioned concurrency and SnapStart both shorten slow starts, but provisioned concurrency keeps environments running ahead of demand and you pay for them, while SnapStart restores a cached snapshot of an initialized environment only when one is needed. Provisioned concurrency is set on a published version or an alias, never on the unpublished $LATEST.

A fixed provisioned amount is usually wrong for traffic with a daily shape. Two CloudWatch metrics show it: ProvisionedConcurrencySpilloverInvocations counts invocations that found no provisioned environment free and ran on standard (cold-start) environments, and ProvisionedConcurrencyUtilization shows how much of the configured amount is in use. High spillover means too little at peak; low utilization means paying for idle environments. Register the alias as a scalable target in Application Auto Scaling and combine:

• a target tracking policy on the predefined LambdaProvisionedConcurrencyUtilization metric (for example, a target of 0.6), which adds and removes environments as utilization moves; and
• scheduled actions that raise the minimum before a ramp you can predict and lower it afterwards, because target tracking reacts to load and needs time to initialize new environments.

Lambda SnapStart takes a snapshot of the memory and disk state after initialization when you publish a version, then resumes new environments from that snapshot. It is available for Java 11 and later, Python 3.12 and later and .NET 8 and later, works only on published versions (and aliases that point to them), and cannot be combined with provisioned concurrency, Amazon EFS or ephemeral storage above 512 MB. For Java there is no extra SnapStart charge. For Python and .NET, a snapshot caching charge (based on memory) is billed for every published SnapStart version for as long as it stays active, whether or not it receives traffic, and each restore is charged; the duration of initialization code and runtime hooks is billed too. That is still different from provisioned concurrency, which bills for the configured number of running environments.

Priming makes SnapStart restores faster still. Work that happens only on the first request (class loading, JSON mapper set-up, building SDK request classes) is not in the snapshot unless it ran before the snapshot was taken. Running those code paths in a beforeCheckpoint runtime hook puts them in the snapshot. The afterRestore hook runs after each restore, so work placed there is repeated on every new environment; use it for state that must be unique per environment, such as refreshing credentials, random seeds or connections.

Amazon EC2 Auto Scaling adds and removes instances so that capacity follows demand, and the policy type decides how much configuration that takes. A target tracking policy is the simplest: you name a metric and a target value (for example, average CPU utilization of 50%), and Auto Scaling creates and manages the CloudWatch alarms and adjusts capacity in both directions.

A warm pool answers slow-booting instances: the bootstrap work (downloading reference files, building a local cache) runs once when the warm instance is created, and the stopped instance then starts quickly when the group scales out. Stopped instances in a warm pool cost only their attached storage (and any Elastic IP addresses), so it is cheaper than raising the minimum number of running instances.

Queue workers need a queue-based metric. When workers spend their time waiting on another service, CPU stays low while the SQS backlog grows, so a CPU target never scales out. The documented pattern is a custom backlog per instance metric: ApproximateNumberOfMessagesVisible divided by the number of in-service instances, published to CloudWatch and used by a target tracking policy. The target value is the acceptable latency divided by the average processing time per message: if a message may wait 45 seconds and each takes 0.5 seconds, each instance can carry 90 messages.

Thread and connection pools improve concurrency inside one process by reusing a bounded set of expensive resources instead of creating a new one for every request. Unbounded creation looks fast under light load and fails under heavy load.

• Bounded thread pools. Creating a thread per request lets a traffic peak start thousands of threads, each with its own stack, until the JVM runs out of memory. A fixed-size pool from an ExecutorService with a bounded work queue keeps parallelism while capping memory, and the queue (or a rejection policy) applies back-pressure.
• Database connection pools. Opening and closing a database connection per request adds TCP, TLS and authentication set-up to every request. A connection pool library in the application keeps connections open and lends them out, without another AWS service. For Lambda, where many short-lived environments each hold connections, Amazon RDS Proxy pools connections on the database side (covered in the Lambda development lesson for Task 1.2).
• SDK HTTP connection pools. Each AWS SDK client keeps its own HTTP connection pool. In boto3 the default max_pool_connections is 10, so when more threads than that share one client, they wait for connections and urllib3 warns that the pool is full and discards the extra connections. Share one client across the threads (boto3 clients are thread-safe) and size its pool to the thread count with botocore.config.Config(max_pool_connections=...).

import boto3
from botocore.config import Config

s3 = boto3.client("s3", config=Config(max_pool_connections=25))  # 25 worker threads

In Lambda, create SDK clients and database connections outside the handler. Code in the initialization phase runs once per execution environment, and later invocations in the same environment reuse the client and its open connections. Creating a client inside the handler repeats that set-up on every invocation.

Lambda has one performance dial, memory, and it also sets compute power: CPU is allocated in proportion to the memory setting. At 1,769 MB a function has the equivalent of one full vCPU, and the maximum of 10,240 MB brings up to 6 vCPUs. Memory runs from 128 MB to 10,240 MB.

• CPU-bound, single-threaded code speeds up with memory until about one vCPU; beyond that a single thread cannot use more.
• CPU-bound, multi-threaded code only runs in parallel when there are several vCPUs. Four worker threads at 768 MB share less than half a vCPU and simply take turns; only enough memory for several vCPUs (about 1,769 MB per vCPU) lets them run at once. Memory is the only setting that changes CPU: ephemeral storage sizes /tmp disk and the timeout sets the time limit.
• I/O-bound code, which mostly waits on DynamoDB or an HTTP API, rarely gets faster with more memory.

Cost is memory multiplied by duration (GB-seconds) plus a per-request charge. Raising memory can lower cost when duration falls by more than the memory rises. Lowering memory saves money when duration stays the same: a function set to 3,008 MB that peaks at 250 MB and runs just as fast at 1,024 MB costs about a third as much per invocation at 1,024 MB. Find the balance by measurement, not guesswork: the open-source AWS Lambda Power Tuning tool is an AWS Step Functions state machine that invokes a function at several memory sizes with a payload you supply and charts cost against duration. It needs no production history, so it suits a function before launch.

Architecture is the second lever. Functions can run on x86_64 or arm64 (AWS Graviton), and AWS prices arm64 duration lower, for better price performance in many workloads. Pure Python, Node.js or Java code usually moves unchanged, but dependencies with compiled native extensions must be built for arm64: a package built on an x86_64 machine fails to import on arm64 with errors such as invalid ELF header. Rebuild the package with the arm64 wheels (for example, in an arm64 build container or with pip install --platform manylinux2014_aarch64 --only-binary=:all:). Memory and runtime changes do not fix a wrong-architecture binary.

AWS Compute Optimizer recommends better-sized resources from each resource's actual CloudWatch utilization history. After you opt in, it analyzes EC2 instances, Auto Scaling groups, EBS volumes, Lambda function memory, Amazon ECS services on AWS Fargate (task CPU and memory) and other resource types, and reports over-provisioned, under-provisioned and optimized resources with the projected savings or risk. (Cost Explorer rightsizing recommendations cover EC2 only.)

Two settings decide how good the recommendations are:

Compute Optimizer needs a minimum amount of metric history before it recommends anything, so it cannot size a function that has never run; use Power Tuning there. Review a "smaller instance" recommendation against peaks and memory before acting on it.

Optimizing messaging means sending fewer, fuller API requests and invoking consumers only for work they will keep, because SQS and SNS charge per request and Lambda charges per invocation.

Amazon SQS

For a Lambda event source mapping on an SQS standard queue, the batch size can go up to 10,000 messages, and a batching window (MaximumBatchingWindowInSeconds, up to 300 seconds) lets Lambda wait to fill a batch. Any batch size above 10 requires a batching window of at least 1 second. When small messages trickle in and invocations carry only one or two, a larger batch size plus a batching window that fits the allowed delay cuts the number of invocations sharply. The batch is invoked when the window ends, the batch size is reached or the payload reaches 6 MB, whichever comes first. FIFO queues keep a maximum batch size of 10 and do not support a batching window.

Amazon SNS fan-out

• Buffer with SQS. An SNS topic invoking Lambda directly invokes it asynchronously, so concurrency follows the publish rate and can overwhelm a downstream database; repeated failures are eventually dropped or dead-lettered. Subscribe an SQS queue to the topic and consume it with an event source mapping whose maximum concurrency caps the pollers' invocations: messages wait in the queue instead of being lost.
• Batch publishing. PublishBatch sends up to 10 messages to a topic in one request. Each message is still delivered to subscribers individually, so subscribers see no change.
• Filter as early as you can. An SNS subscription filter policy stops unwanted messages before they reach the queue, but it is set on the subscription, which belongs to the topic owner. If you own only the queue and the function, add filter criteria to the Lambda event source mapping: Lambda evaluates each message and invokes the function only for matches. For SQS, messages that do not match are deleted from the queue, so filter only on messages you truly want to discard.

{"Filters": [{"Pattern": "{\"body\": {\"status\": [\"SHIPPED\"]}}"}]}

Body filtering works on JSON bodies; with raw message delivery off, the body is the SNS envelope and the original message sits inside its Message string.

A CloudFront cache works well when many requests share one cache key and objects stay cached long enough to be reused; the cache policy controls both the cache key and the TTLs.

Cache key versus what the origin receives

Everything in the cache policy's key (query strings, cookies, headers) splits the cache: including all cookies, all query strings or User-Agent creates a separate cached copy per user or device, and the hit ratio collapses. Put only the values that actually change the response in the cache policy. Values the origin needs for other reasons (a session cookie for logging, an authorization header) go in an origin request policy, which forwards them to the origin without adding them to the cache key. Values in the cache key are forwarded automatically.

TTL settings

Releases without invalidations

Invalidations remove objects from edge caches, but they take time, are charged per path beyond a monthly free allowance and do not clear copies already in browsers. The durable pattern for single-page apps is versioned file names: the build adds a content hash to each JS and CSS file name, so those files can be cached for a year (Cache-Control: max-age=31536000, immutable) and a new release simply references new names. The entry point, index.html, is served with no-cache so browsers revalidate it on each load and pick up the new names at once.

CloudFront publishes a CacheHitRate metric (one of the distribution's additional metrics) to confirm a change worked.

A cache helps only while it can hold the working set and the requests can be answered from it, and CloudWatch metrics show which of those is failing. Cache patterns themselves (lazy loading, write-through, choosing an engine) are covered in the data stores lesson for Task 1.3; this section is about tuning a cache that already exists.

Amazon ElastiCache for Valkey and Redis OSS

When evictions climb, memory sits near 100% and CPU is low, the fix is more memory: scale up to a node type with more RAM, or, for Valkey and Redis OSS, enable cluster mode and add shards so the data set is spread out. High EngineCPUUtilization with spare memory points to a node with more compute or more shards instead. Memcached reports a different metric set (for example, CPUUtilization and BytesUsedForCacheItems) and scales out by adding nodes.

DynamoDB Accelerator (DAX)

DAX is an in-memory, DynamoDB-compatible cache for read-heavy workloads that accept eventually consistent reads, returning cached items in microseconds. Its limits matter:

• Reads with ConsistentRead=true and transactional reads (TransactGetItems) are passed straight through to DynamoDB and are not cached. Code that sets strong consistency on every call gains nothing from DAX: if the data may be a few seconds stale, send eventually consistent reads through the DAX client.
• Items stay in the item cache for a TTL set in the cluster's parameter group (5 minutes by default).
• Writes go through DAX to DynamoDB (write-through), so DAX does not reduce write cost or write throttling.

A bottleneck is found by measuring the slow part directly: the right statistic of the right metric, then the component behind it. Setting up alarms, dashboards and tracing is covered in Task 4.2, and log-based root-cause analysis in Task 4.1; this section is about reading the data for optimization.

Metrics that expose bottlenecks

• Percentiles, not averages. An average can look healthy while a small share of requests is many times slower. Graph a percentile such as p99 (or p95/p90) to see the slow tail and track it over time.
• Queue lag. For SQS consumers, ApproximateAgeOfOldestMessage shows how long the oldest message has waited (compare it with the processing deadline) and ApproximateNumberOfMessagesVisible shows the backlog waiting to be received. Both rising together means consumers are falling behind. ApproximateNumberOfMessagesNotVisible counts in-flight messages, ApproximateNumberOfMessagesDelayed counts delayed ones and NumberOfEmptyReceives measures polling waste.
• Lambda Insights. The Lambda Insights extension (a layer) adds system metrics such as CPU time, memory used and network for each function, which shows whether a function is CPU-bound, memory-bound or waiting.
• DynamoDB hot keys. Throttling while consumed capacity is well below provisioned capacity usually means a few partition keys get most of the traffic. CloudWatch Contributor Insights for DynamoDB, enabled per table or index, shows the most accessed and most throttled keys with no code changes.

AWS AI tools for optimization

AI coding assistants can find optimization opportunities faster, provided they see real data. Kiro connects to tools through the Model Context Protocol (MCP): adding the open-source AWS CloudWatch MCP server to Kiro's MCP configuration (.kiro/settings/mcp.json in the workspace, or ~/.kiro/settings/mcp.json for the user) lets the agent query metrics, active alarms and CloudWatch Logs Insights itself while it reads the code.

Typical AI-suggested fixes are code-efficiency changes, such as replacing a loop of GetItem calls with one BatchGetItem (up to 100 items or 16 MB per request), and resource changes, such as lowering an over-provisioned memory setting. Accept a suggestion only when it keeps behaviour and the evidence supports it:

• BatchGetItem can return some keys in UnprocessedKeys (for example, when throttled or over the size limit). Those items were not read; they are not missing. Resend them, with exponential backoff, until none remain.
• Check a memory or concurrency suggestion against the stated goal. A setting change that does not shorten measured duration or latency only adds cost, and one that keeps duration the same at a lower setting saves money.