Operational Efficiency and Optimization for GenAI Applications practice questions

Response limiting and quota: at request start Bedrock deducts total input tokens plus maxTokens from TPM/TPD, then adjusts to actual output when the request finishes.

The symptom is throttling while processed tokens are low, which points to the up-front reservation. Each request reserves input + maxTokens, so a needlessly large maxTokens lets far fewer requests start per minute. Setting maxTokens slightly above the longest observed output keeps responses complete and multiplies the concurrent requests that fit in the same quota at no cost. Priority and Provisioned Throughput cost more; retries don't change the reservation.

1. Compare the throttling with the tokens actually processed.
2. Recall that maxTokens is deducted from the quota when a request starts.
3. Size maxTokens to observed output with headroom.
4. Reject options that add cost or only retry.

Exam tip: Oversized maxTokens wastes quota: Bedrock reserves input + maxTokens up front, so size it to real outputs.

Capacity planning for token quotas: quota usage per request = InputTokenCount + CacheWriteInputTokenCount + OutputTokenCount x burndown rate; billing counts tokens 1:1.

Bedrock first deducts input + maxTokens, then adjusts during processing and settles at input + output x burndown, returning any unused part. For a model with a 5x output burndown, each request settles at 2,000 + 1,500 = 3,500 quota tokens, so 600 requests need about 2.1 million TPM. The billed figure (1.38 million) and the initial deduction (1.44 million) both understate it; applying the burndown to maxTokens (2.4 million) overstates it.

1. Write the settled quota formula: input + cache writes + output x burndown.
2. Apply it per request: 2,000 + 300 x 5 = 3,500.
3. Multiply by 600 requests per minute.
4. Check the distractors against billing and initial-deduction formulas.

Exam tip: Plan TPM with the burndown rate: quota tokens per request = input + output x burndown, not the billed token count.

Response size controls: output tokens are billed as generated, so limit length in the request (instruction plus maxTokens), not after the response arrives.

Cost follows generated tokens. Telling the model the target length produces a complete short text, and maxTokens set just above it is a safety cap. A tight maxTokens alone cuts text mid-sentence, temperature does not control length, and trimming after generation pays for every token anyway.

1. Identify the cost driver: generated output tokens.
2. Find a control that shortens generation, not display.
3. Check that the text stays complete.
4. Combine a length instruction with a maxTokens cap.

Exam tip: Shorten outputs at generation time: ask for the length you need and cap maxTokens just above it.

Response limiting with stop sequences: generation ends when the model produces a configured string, which cuts unwanted trailing output exactly where the format defines its end.

The template already marks the end of the useful output with </category>. Adding it as a stop sequence ends generation there, removing about 150 billed tokens per call; the response ends with stopReason stop_sequence and does not include the matched tag, so the parser only needs to read the text after <category>. A tiny maxTokens risks truncating longer category names, topP does not control length, and JSON output still contains the justification.

1. Find where the useful output ends in the format.
2. Pick a control that stops generation at that point.
3. Check that varying category lengths are not truncated.
4. Reject options that still generate the justification, and note that the stop sequence itself is not returned.

Exam tip: When the useful answer ends at a known delimiter, make that delimiter a stop sequence.

Context window optimization for chat: keep a sliding window of recent turns and replace older turns with a summary of the facts that must persist.

Input cost grows because every turn resends everything. Keeping recent turns plus a summary bounds the prompt while keeping facts such as the account number. A plain window loses those facts, a bigger context window changes capacity not cost, and dropping history entirely breaks the conversation.

1. Identify why input tokens grow per turn.
2. List what must survive pruning: early key facts.
3. Combine recent turns with a summary of older ones.
4. Reject options that lose facts or don't reduce tokens.

Exam tip: Prune chat history with a sliding window plus a running summary of facts that must persist.

Context pruning for agents: tool outputs accumulate in the history and are resent on every model call, so trim them at the source and compact them once used.

In a tool loop the history is resent on each call, so a 6,000-token result is paid for again and again. Returning only the needed fields cuts it at the source; summarizing used results compacts what remains. A larger context window doesn't reduce tokens, a prompt cache checkpoint makes the resent history cheaper to read but still sends every token, and temperature does not control the number of tool calls.

1. Find what grows: tool results accumulating in the message history.
2. Shrink the results where they are produced.
3. Compact results that have already served their purpose.
4. Reject options that change the price or limit of the tokens rather than the number sent.

Exam tip: In agent loops, trim tool payloads to the fields used and summarize stale tool results.