Design Resilient Architectures practice questions

Guaranteeing no lost work between a producer and an overwhelmed consumer requires two things together: a durable buffer that owns each message until it is processed, and a consumer tier that scales on the buffer's depth.

The queue and the queue-driven Auto Scaling group are two halves of one design. SQS satisfies the no-loss requirement because a message survives consumer failure and reappears after the visibility timeout until a worker deletes it post-success. Scaling fulfillment on queue depth satisfies the elastic-capacity requirement without human action. SNS fails the no-loss requirement on its own: it is push-based, retains nothing for consumer-paced retrieval, and exhausts retries against a saturated endpoint. Client-side exponential backoff keeps the tight coupling; the order lives in volatile memory during retries, so the failure window is narrowed but not closed, and it adds latency to the intake path during exactly the traffic peak. Enlarging the database is vertical scaling aimed at an unstated bottleneck; the stem describes a coupling failure, not a database failure.

1. Restate the two hard requirements: zero lost orders and automatic capacity growth.
2. Map zero loss to durable buffering: only SQS holds an order independently of both the API and fulfillment being healthy.
3. Map automatic growth to a scaling signal: queue depth is the direct measure of unprocessed fulfillment work.
4. Eliminate SNS push and client retries because both leave a window where the only copy of an order can vanish.
5. Eliminate database resizing because it neither buffers nor scales the failing tier.

Exam tip: No-lost-work plus elastic capacity is always the pair: SQS as the durable buffer, workers scaling on queue depth.

SQS FIFO queues exist for exactly two guarantees standard queues cannot make: strict ordering within a message group and exactly-once processing within the deduplication window. The message group ID scopes ordering, enabling parallelism across groups.

The scenario states both FIFO trigger conditions explicitly: strict per-account order and no duplicates. Using the account ID as the message group ID gives each account its own ordered lane while different accounts process concurrently, which also addresses the parallelism requirement. The idempotent-consumer option is the strongest distractor because idempotency genuinely solves the duplicate half, but it cannot restore ordering that a standard queue never promised, so it fails half the requirement. Long polling is a cost and latency optimization for retrieving messages; it is orthogonal to delivery semantics. Standard SNS fails on all counts: unordered, at-least-once push delivery with no queueing for the consumer to pace itself.

1. Extract the requirements: strict order per account, no duplicate processing, parallelism across accounts.
2. Match strict ordering and exactly-once to FIFO, the only queue type that provides them.
3. Scope ordering with the message group ID set to the account ID so unrelated accounts do not serialize behind each other.
4. Reject standard-queue options because best-effort ordering violates the per-account sequence requirement no matter how consumers are written.

Exam tip: Order plus no-duplicates means FIFO, and the message group ID is how you get per-entity ordering with cross-entity parallelism.

Default to SQS standard and choose FIFO only when the workload genuinely requires strict ordering or exactly-once processing. FIFO buys those guarantees by capping throughput; standard queues scale nearly without limit.

The scenario removes both reasons FIFO exists: order does not matter, and duplicates are already handled by idempotent consumers. That leaves throughput and simplicity as the deciding factors, and a single standard queue delivers effectively unlimited throughput with nothing extra to build or operate. The single FIFO queue is the safety-blanket trap the stem sets up: choosing guarantees you do not need buys a hard throughput ceiling. The sharded-FIFO option is the technically-workable-but-inferior design; it reintroduces the throughput via complexity, adding a routing layer, more queues, and more failure modes, which loses under LEAST complexity. Amazon MQ is for lift-and-shift of applications speaking broker protocols like AMQP or JMS; for a cloud-native firehose it brings broker capacity management without matching standard SQS scale.

1. Check the FIFO triggers: no ordering requirement and duplicates already tolerated, so FIFO is unjustified.
2. Recall the cost of FIFO: capped throughput versus nearly unlimited for standard.
3. Compare the workable options against the qualifier: one standard queue versus a sharded FIFO fleet with routing code; the simpler design wins.
4. Confirm consumers being idempotent makes at-least-once delivery fully acceptable.

Exam tip: Picking FIFO without an ordering or dedup requirement buys a bottleneck; standard plus idempotent consumers is the high-throughput default.

SQS does not delete a message on receive; it hides it for the visibility timeout while the consumer works, and the consumer deletes it after success. A timeout shorter than processing time makes healthy in-flight messages reappear and be processed again.

The symptom signature is exact: duplicates without worker crashes means the visibility timeout is expiring before processing completes, so the message becomes visible again and another worker picks it up. Raising the timeout above the maximum processing time (with margin) removes the premature reappearance and the duplicates with it. The FIFO conversion is the classic trap: FIFO's exactly-once guarantee deduplicates producer-side sends, but visibility-timeout expiry is a consumer-side redelivery that FIFO performs as well, so the problem would persist with a lower throughput cap added. Long polling addresses a completely different concern, the cost and latency of polling an empty queue. A dead-letter queue catches poison messages that keep failing; here every processing attempt succeeds, so the redrive counter is irrelevant to the duplication.

1. Note the two facts: processing takes up to five minutes and visibility timeout is 30 seconds.
2. Recall the mechanism: an in-flight message reappears when its visibility timeout expires before the consumer deletes it.
3. Conclude the duplicates are premature redeliveries, not producer duplicates or failures.
4. Fix the mismatch by raising the visibility timeout above the worst-case processing time rather than changing queue type or adding a DLQ.

Exam tip: Messages processed twice by healthy workers means the visibility timeout is shorter than processing time; raise the timeout, do not switch to FIFO.

A dead-letter queue with a redrive policy is the built-in mechanism for poison messages: after maxReceiveCount failed receives, SQS moves the message aside automatically so it can be analyzed without blocking the main queue.

The redrive policy is a queue configuration, not code, and it does exactly what the requirement states: isolate repeatedly failing messages automatically and preserve them for analysis, with an alarm providing visibility. Deleting on exception is the tempting code fix, but it destroys the malformed messages the team wants to analyze, cannot distinguish a poison message from a transient failure, and pushes failure policy into every consumer. Purging is a destructive manual action that sacrifices all valid in-flight work to remove a few bad messages. Raising the visibility timeout misreads the mechanism: the messages are not duplicating prematurely, they are genuinely failing, so slowing the retry loop just delays the same infinite cycle.

1. Identify the pattern: messages that always fail and recycle forever are poison messages.
2. Recall the built-in answer: a redrive policy with maxReceiveCount moves them to a DLQ automatically.
3. Verify the requirement fit: DLQ preserves the messages for analysis and unblocks the main queue with zero ongoing human effort.
4. Eliminate destructive options (delete-on-failure, purge) because they lose data, and timeout tuning because the messages fail rather than duplicate.

Exam tip: Messages that repeatedly fail belong in a dead-letter queue via a redrive policy; never delete or purge your way around a poison message.

One event consumed independently by many systems is the fan-out pattern: an SNS topic delivers a copy to every subscriber, and subscribing an SQS queue per consumer adds durable buffering, independent retries, and independent scaling.

SNS-to-SQS fan-out satisfies all three requirements at once: every subscriber receives its own copy (unlike a shared queue), each queue holds the backlog while its consumer is down (unlike direct push or synchronous calls), and adding a consumer is a subscription change that never touches the publisher. The shared single queue is the classic elimination: competing consumers each steal messages, so inventory would see only a fraction of the events. The three-queue publisher option is the strongest distractor because it functions, but it moves fan-out logic into application code, forcing the publisher to handle partial failures across three sends and to change whenever a consumer is added, which violates the stated extensibility requirement. Direct synchronous calls are tight coupling and lose events outright when a consumer is offline.

1. Identify the delivery semantics needed: every consumer must get every event, which means pub/sub, not competing consumption.
2. Add durability per consumer: subscribe an SQS queue for each system so downtime becomes a backlog instead of loss.
3. Check extensibility: new consumers are new subscriptions, leaving the ordering application untouched.
4. Eliminate the shared queue (message stealing), publisher-managed multi-queue writes (coupling and code change per consumer), and synchronous calls (availability coupling).

Exam tip: Every-consumer-gets-a-copy plus per-consumer durability is SNS fanning out into one SQS queue per subscriber.