AIP-C01 Prompt Engineering and Prompt Governance on Amazon Bedrock

Prompt engineering and governance is the discipline of controlling what a foundation model (FM) is told, how it must answer, which rules are enforced regardless of the prompt, and who may change, release and use each prompt. Task 1.6 treats prompts as production assets: they live in a managed store, are versioned and approved like code, are tested before release, are watched for regressions after release, and are audited. Six skills make up the task, and each has a typical set of AWS tools.

Neighbouring tasks own related material: choosing and configuring the model is Task 1.2, building Converse requests in general is Task 1.3, prompt-injection and content/PII filtering are Domain 3, and model evaluation jobs and LLM-as-a-judge evaluation are Task 5.1. This lesson uses those features only where they serve prompt control and governance.

Amazon Bedrock Prompt Management stores a prompt as a managed resource, so the role, tone, refusal wording and output instructions are defined once and every application calls the same definition by ARN. Hard-coding a system prompt in each service is how products drift apart. As a general rule, a plain configuration store holds text but not variables, model settings, console testing or versions, and an agent is orchestration you add only when the task needs tools or planning, not to share instructions.

What a managed prompt holds

• Template and variables. The text contains placeholders such as {{product_name}} or {{region_policy}}, filled at runtime. Anything that differs per call, tenant or product belongs in a variable, not in a separate copy of the prompt.
• Template type. A text template is a single prompt. A chat template has a system prompt plus messages (and optional tool configuration), which suits multi-turn assistants: the role lives in the managed system prompt, and the conversation's earlier user and assistant turns are appended at runtime as real messages.
• Model and inference configuration. The model ID and settings such as temperature and maximum tokens are part of the prompt.
• Variants. Authors compare alternative wordings or models side by side in the console test window before choosing one.

Calling a managed prompt at runtime

Pass a prompt version ARN as the modelId (the Converse reference asks for the version ARN; production should always pin one) of a Converse (or InvokeModel) request and supply promptVariables to fill the placeholders. You may append messages, such as earlier conversation turns. You may not send system, inferenceConfig, toolConfig or additionalModelRequestFields: those must be defined in the prompt itself, and including them returns a ValidationException. The fix for a new setting or per-tenant rule is therefore to change the prompt (for example, set the temperature or add a variable such as a region policy), publish a new version and fill the variable per call. The caller needs permission for bedrock:RenderPrompt as well as the model invocation permission.

A system prompt is guidance the model can be talked out of; a Bedrock guardrail is a policy evaluated by a separate service on the user input and on the model output, whatever the prompt or the conversation says. Role-play and rephrasing routinely defeat "never do X" instructions, so a rule that must hold no matter the wording belongs in a guardrail (for example a denied topic), and the instructions only shape style and behaviour within it.

As a rule, any custom filter you build yourself covers only the side it is placed on (input or output) and is code you maintain, whereas a guardrail screens both with configuration alone. Guardrails do not replace deterministic business checks either; a word filter or grounding check cannot count bullet points or add up line items (see the quality assurance section).

Ways to apply a guardrail

• Per request: pass guardrailConfig (guardrail ID and version) on Converse or InvokeModel.
• Inside a flow: add a guardrailConfiguration to a prompt node, which screens that node's input and response.
• Without a Bedrock model: the ApplyGuardrail API is decoupled from foundation models. For a model on a SageMaker AI endpoint or elsewhere, call it with source INPUT on the user's text before invoking the model and with source OUTPUT on the response; when the action is GUARDRAIL_INTERVENED the result carries the guardrail's configured blocked message. The heavier path, moving the model into Bedrock with Custom Model Import so that guardrailConfig can be passed on each request, means re-packaging and re-hosting the model only to gain a filter that ApplyGuardrail already provides beside the existing endpoint.
• Account-level enforcement: an enforced guardrail configuration (PutEnforcedGuardrailConfiguration, set per account and Region) applies a guardrail to invocations in that account even when requests omit it, but it stays in that account administrator's hands.
• Organization-level enforcement: an AWS Organizations Bedrock policy attached to the root, OUs or accounts references a versioned guardrail in the management account that is shared through a resource-based policy granting bedrock:ApplyGuardrail. It applies to every invocation in scope, needs no application change and cannot be removed by member-account administrators. Organization, account and request guardrails combine, so teams can add stricter guardrails of their own.

Reliable formats come from three layers: instructions and templates that describe the format, structured outputs that constrain the model to a JSON Schema, and a validation step for rules a schema cannot express. Instructions and examples raise consistency but guarantee nothing, and sampling or stopping settings change how text is generated, not whether it matches a schema.

Converse structured outputs

On supported models, Converse accepts a JSON Schema in outputConfig.textFormat with type json_schema. Bedrock compiles the schema into a grammar and constrains decoding, so the response parses and contains the required fields with no parse-and-retry loop. Strict tool use (a tool definition with strict mode) uses the same mechanism and the same schema subset. Know the rules:

• Every object must set additionalProperties to false.
• The supported subset excludes numeric and length constraints such as minimum, maximum, multipleOf, minLength and maxLength. A schema that uses them fails with a 400 ValidationException on every request. Remove them (you can describe the limit in the property's description) and enforce the limit after the response.
• Constrained decoding does not lift the token limit. If stopReason is max_tokens, the JSON was cut off mid-way. Give long inputs a higher maxTokens and check the stop reason before parsing so truncated responses are retried rather than breaking the parser.

Shape versus meaning

A clarification workflow pauses when the model finds information missing, asks the user, and resumes with the answer; in Step Functions this is a Standard workflow Task state that uses the wait-for-callback pattern (.waitForTaskToken). The task passes a token to the step that sends the question (for example a Lambda function or an SNS/SQS message to the app), and the execution stops without consuming compute. When the user replies, the reply handler calls SendTaskSuccess with the saved token and the reply, and the execution resumes with its earlier state intact. A Standard execution can wait days.

So an Express workflow that must wait for a person has to become a Standard workflow. The general rule for waiting: anything that checks repeatedly is polling, and anything that holds compute open while waiting is billed for idle time and bounded by that compute's own limit (15 minutes for Lambda). The callback pattern does neither.

Looping until the answer is complete

Model the loop on the business result, not on errors. Keep a round counter in the state data; after each reply re-run the extraction, then a Choice state checks whether fields are still missing and how many rounds have passed, looping back to ask again or routing to a human handoff.

Interactive assistants route each message by intent and carry context between turns: Amazon Comprehend custom classification recognises the intent that selects a prompt, and DynamoDB stores the conversation so each model call gets the history it needs.

Intent recognition with Amazon Comprehend

Comprehend custom classification trains a classifier from your labelled examples with no model-training code, and a real-time endpoint classifies each message synchronously (endpoint throughput is set in inference units). Generic pretrained features such as key phrases or entities do not know your business intents, and a custom entity recognizer extracts spans of text rather than labelling a whole message.

Every classification result includes confidence scores, which an application can compare with a threshold of its choosing: act on confident results and send low-confidence messages down a clarifying-question path. In general, behaviour you control in application code changes at once, while anything that changes the classifier itself (its mode, its classes, its training data) means training a new model version; capacity settings change throughput only.

Conversation history in DynamoDB

• Key design: session ID as the partition key and turn timestamp (or sequence) as the sort key, so a Query with descending order and a limit reads the latest turns directly. A timestamp partition key forces Scans, and appending every turn to one list attribute eventually hits the 400 KB item limit.
• Expiry: write an expiry timestamp on each item as it is stored and enable TTL on that attribute; DynamoDB deletes expired items in the background without consuming write capacity, so no cleanup job is needed.
• Bounding the context: sending the whole transcript makes input tokens grow every turn and eventually exceeds the context window. Send the last few turns verbatim plus a rolling summary of older turns that the model refreshes periodically and DynamoDB stores with the session.

Prompt governance means production runs exactly the prompt that was approved, only authorised people can release a prompt, every change has history, and both prompt usage and model traffic are recorded. Four mechanisms combine.

Versions as the release unit

A Prompt Management version is an immutable snapshot of the prompt's working draft. Production passes a version ARN as the modelId, so daily draft edits never reach it, and rollback is switching to an earlier version's ARN. Prompt variants are alternatives inside one prompt for comparison, not releases.

Approval gates with IAM

Creating a version is its own IAM action, bedrock:CreatePromptVersion. Allow authors to create, update and test prompts (UpdatePrompt, RenderPrompt) but deny them CreatePromptVersion; grant it to a reviewer role (or a pipeline the reviewer approves) and point production at the version it creates. The principle: gate the action that creates the release, not the actions authors need every day.

Template repositories in Amazon S3

When templates live as files in S3 before import, enable S3 Versioning so every overwritten template stays as a noncurrent version that can be restored, and use S3 event notifications for object-created events (to Lambda, SQS, SNS or EventBridge) to start validation on each upload.

Tracking usage and access

CloudTrail never stores prompt or response text, so "who used which prompt or flow" without content is a data-event question, while "searchable full text for a year" is invocation logging with a log-group retention setting and Logs Insights. CloudTrail Insights only flags unusual call volumes.

Prompt quality assurance checks every output before it is used, tests each prompt version against edge cases before release, and watches production for regressions after release, using Lambda, Step Functions and CloudWatch respectively.

Verifying each output with Lambda

Put a Lambda function after the model step to test deterministic rules. Typical rules: a required number of list items, a length budget, a mandatory field value, line amounts that add up to a stated total. Have it return a verdict naming the broken rule rather than throwing an error. A Choice state with an attempt counter can then call the model again with the failed rule added to the prompt, and send repeat failures to a human review queue (for example SQS). Every attempt stays visible in the execution history. Keeping the loop in the state machine rather than inside one function is what makes each attempt auditable and bounded.

Edge-case suites with Step Functions

• Distributed Map reads a large dataset directly from S3 (for example a JSON Lines file), runs a child workflow per item at a chosen MaxConcurrency, writes results with ResultWriter and can stop the run at a tolerated failure percentage. The workflow then computes the pass rate and blocks approval below the gate.
• Inline Map takes its items from the state input, so it is bound by the 256 KiB payload limit and runs at most 40 iterations concurrently.
• The optimized integration arn:aws:states:::bedrock:invokeModel calls a model straight from a Task state, with input or output optionally in S3, so the suite needs no Lambda or container code for model calls.
• Throttling (ThrottlingException) means the suite is faster than your quota: lower MaxConcurrency and add a Retry with exponential backoff and jitter. Raising the tolerated failure percentage hides untested items.
• Bedrock batch inference is the cheaper way to run very large offline prompt sets, but jobs are asynchronous and queued with no guaranteed completion time, so it does not suit a time-boxed release gate that must return a pass rate before approval.

Regression detection with CloudWatch

Built-in Bedrock metrics (invocations, latency, token counts, throttles) are per model ID and say nothing about answer quality. Publish your own quality signal as a custom metric with dimensions identifying the prompt and its variant (for example PromptId and Variant) from the code that already grades responses, and alarm on it with anomaly detection, which learns the metric's normal band and alerts on a clear drop. When live traffic is too low to measure, a CloudWatch Synthetics canary sends golden inputs on a schedule, asserts on the answers, stores run history and publishes SuccessPercent to alarm on, with no scheduling or reporting code to write.

Iterative refinement improves response quality by structuring the input, specifying the output, asking for explicit reasoning, and closing the loop with measured feedback rather than guesswork.

Structured input components

Wrap each component (document, instructions, question, examples) in its own XML-style tag and refer to those tags in the instructions, so the model cannot confuse a contract clause with the question. For long documents, place the document first and the question and instructions after it. Retrieval is no substitute when the whole document must be considered. In Converse, document content blocks belong to user messages, not the system field.

Chain-of-thought with a parseable answer

Multi-step problems improve when the model reasons before answering. Ask for the reasoning inside one tag (for example <thinking>) and the result inside another (for example <output>). The application extracts only the answer tag, so downstream parsers get a stable location and users never see the reasoning. The general rule: reasoning has to be produced to help, so hiding it is a parsing job for the application, not something to forbid in the prompt; examples teach whatever they show, and inference parameters change randomness, not method.

Feedback loops

• Tag every request with its prompt version and a per-answer ID in the Converse requestMetadata field; the tags appear in the model invocation log record next to the full text.
• Store each user rating with the same answer ID and prompt version, so you can compute low-rating rates per version and pull the low-rated prompts and responses from the logs.
• Turn low-rated cases into a test set, compare candidate variants against the current version on it, and release the winner as a new version. Never auto-rewrite production prompts or ship a rewrite untested.

Fine-tuning changes the model, not the prompt, so it is outside prompt refinement.

Amazon Bedrock Flows (formerly Prompt Flows) link prompts, logic and AWS services into a visual workflow, giving you sequential prompt chains, conditional branches on model output, loops, reusable prompt components and built-in pre- and post-processing.

Versions and aliases

You edit a flow's working draft and test it through the built-in test alias (TSTALIASID), which always tracks the draft. A version is an immutable snapshot; an alias points at a version, and applications invoke the flow through an alias. Releasing means repointing the alias, and rollback means repointing it to the previous version, with no code change and without exposing unfinished draft edits. Invocations of a flow alias are CloudTrail data events on AWS::Bedrock::FlowAlias.