AIP-C01 Input and Output Safety: Bedrock Guardrails, Prompt Injection Defence and Hallucination Checks

The managed way to stop harmful or out-of-scope user input is an Amazon Bedrock guardrail: a reusable configuration of policies that evaluates the prompt before the model runs and, if a policy blocks it, returns a configured message without invoking the model. You attach it to a request with guardrailConfig (Converse/ConverseStream) or the guardrail ID and version (InvokeModel); when it intervenes on Converse, the response carries stopReason guardrail_intervened. Because it is configuration, not code, it beats a custom classifier, a Comprehend-plus-Lambda pipeline or a system-prompt instruction (which the model may simply ignore) whenever the requirement is "managed" or "least operational overhead".

A guardrail combines several policy types, and each answers a different question:

Strength is set per direction. Each content filter has an input strength (prompts) and an output strength (responses): NONE, LOW, MEDIUM or HIGH. Strength decides which confidence levels are blocked: HIGH blocks content classified as harmful with high, medium or low confidence, while LOW blocks only high-confidence content. So if one category over-blocks on one side only (for example, legitimate first-aid questions tripping a harm category on prompts), lower that category's input strength and leave its output strength and the other categories alone. LOW is not off: clear, high-confidence harm is still blocked.

Each filter also has an action: BLOCK, or NONE (shown in the console as Detect (no action)), which evaluates and reports in the trace but lets the content through.

Tiers and languages. Content filters, denied topics and some other policies come in a Classic and a Standard tier. Classic supports English, French and Spanish (word filters are limited to the same three). Standard supports a much longer language list (Japanese and German among the optimised ones), adds capabilities such as prompt leakage detection, and requires the guardrail to use cross-Region inference through a guardrail profile, so data may be processed in other Regions within the same geography. If a scenario needs non-English filtering and accepts cross-Region processing, Standard tier is the managed answer; translating everything to English first works but adds a service and latency.

The cheapest place to reject a bad request is before any compute runs, at the API layer. Two managed controls do this for a GenAI endpoint: API Gateway request validation for the request's shape, and AWS WAF for size and volume abuse.

Request validators (REST APIs only). On an API Gateway REST API method you attach a request validator and a model, which is a JSON Schema for the body (required properties, types, maxLength and so on). A request that fails gets a 400 and the integration (the Lambda function that calls Bedrock) never runs, with no validation code to write. A Lambda authorizer is the wrong tool for this: it receives headers, query strings and context, not the request body, and it is custom code anyway.

So a team that moved to an HTTP API to save money has also given up declarative body validation and WAF; getting them back at the API layer means a REST API.

AWS WAF on the stage. A web ACL associated with a REST API stage blocks requests before they reach the integration. A size constraint rule rejects oversized bodies, and a rate-based rule blocks request floods, aggregating by source IP by default. A usage plan, by contrast, throttles per API key, which does nothing against an anonymous flood from one address.

The body inspection limit is a trap. WAF only inspects the body up to the web ACL's body inspection limit, which defaults to 16 KB for API Gateway (and CloudFront, Cognito and similar resources) and can be raised in 16 KB steps up to 64 KB. Anything beyond the limit is handled by the rule's oversize handling: Continue (inspect only what was read), Match (treat the request as matching the rule) or No match. So a size rule whose threshold is above the inspection limit can never match while oversize handling is Continue: WAF never reads that far into the body. A size rule only works when the inspection limit covers its threshold, and oversize handling decides what happens to everything beyond what WAF read.

When moderation needs more than a guardrail, for example your own scoring rules, several services or a human decision, orchestrate it with AWS Step Functions: Lambda functions score the prompt (often with Amazon Comprehend), a Choice state routes on the score, and borderline content goes to review. The workflow type decides what is possible.

Human review that can take hours or days is a Standard workflow with a .waitForTaskToken task: the task sends a token to your moderation queue or UI, the execution pauses, and it resumes when the reviewer's decision comes back through SendTaskSuccess or SendTaskFailure. A Lambda function cannot wait that long (15-minute limit), and polling a table from Lambda wastes money. Amazon SageMaker Augmented AI (A2I) is no longer open to new customers, so do not design new human-review loops around it.

High-volume, sub-second checks the API must wait for are a synchronous Express workflow, which API Gateway can start directly with StartSyncExecution and which returns the verdict to the caller. At-most-once semantics are fine when re-running a check is harmless.

Both at once. A fast verdict for clear prompts plus a day-long review for borderline ones needs two workflows: a synchronous Express workflow scores and returns the verdict, and for borderline prompts it hands off (a fire-and-forget StartExecution) to a Standard workflow that waits on a task token, notifying the user later. The Express workflow cannot pause on a token itself, cannot use .sync to wait for the child, and a Standard workflow cannot be started synchronously.

A guardrail filters responses with the same policies it applies to prompts, using each policy's output-side settings. If a response is blocked, the user receives the guardrail's blocked-response message; a blocked prompt returns the separate blocked-prompt message (blockedOutputsMessaging and blockedInputMessaging). Setting them to different texts lets you ask users to rephrase their own message but tell them an answer can't be shared, with configuration only. A Converse request carries one guardrail.

Guardrails for any model: ApplyGuardrail. The ApplyGuardrail API evaluates text against a guardrail without invoking a foundation model. You pass the content with source INPUT (a user prompt) or OUTPUT (a model response) and get back an action of GUARDRAIL_INTERVENED or NONE, plus the masked text or blocked message. That reuses one set of policies for a model on a SageMaker AI endpoint, a third-party model, or any pipeline stage, with no change to where the model is hosted. SageMaker's InvokeEndpoint has no guardrail parameter of its own.

Images. Content filters have input and output modalities. Text is on by default; the image modality must be selected explicitly. A guardrail created with default options will block a violent text prompt but let an image model's violent output through until you enable the IMAGE output modality on those filters. Text-only policies such as denied topics do not inspect images.

Custom patterns in output. A sensitive information filter with a custom regex can mask or block patterns such as an organisation's own reference-number formats in responses, with no Lambda code. Comprehend's PII detection does not know an organisation's own ID formats.

Streaming. With ConverseStream or InvokeModelWithResponseStream, the guardrail has a stream processing mode:

If a stream must never show unmasked or harmful content, use synchronous mode. Switching a mask action to block in asynchronous mode still leaks the chunk that triggered detection.

A prompt attack is an attempt to override the application's instructions: a jailbreak (bypass safety behaviour), a prompt injection ("ignore your previous instructions and...") or prompt leakage (extract the system prompt). The guardrail's prompt attack filter detects these on user input. It detects jailbreaks and injection on both tiers; prompt leakage detection is Standard tier only, which in turn needs cross-Region inference. Raising a Classic-tier strength cannot add a category, and word filters for phrases like "system prompt" are trivially evaded.

Input tags tell the guardrail which text is the user's. A developer's own system prompt often reads like an instruction override, so the filter must evaluate only user-supplied text:

• With InvokeModel / InvokeModelWithResponseStream, wrap the user's text in <amazon-bedrock-guardrails-guardContent_suffix> tags and pass the matching tagSuffix. Without input tags, prompt attacks are not filtered on these APIs, although the other filters still evaluate the whole prompt. That is why hate content can be blocked while "ignore your instructions" sails through.
• The tagSuffix (1 to 20 alphanumeric characters) should be a new random value for every request. A static suffix lets a user type the closing tag and append instructions outside the tagged region that the guardrail does not evaluate but the model still reads.
• With Converse, the equivalent is a guardContent block around the user content to be evaluated.

Indirect prompt injection arrives through content the application feeds the model rather than through the user: web pages, retrieved documents, emails, tool output. In the Converse API a guardrail does not evaluate tool results (toolResult), tool definitions or the model's tool-call arguments, and the prompt attack filter documentation says the same. To catch instructions hidden in fetched content, screen it yourself before it reaches the model: call ApplyGuardrail with source INPUT on each tool result and return an error result (or drop the content) when it intervenes. System-prompt instructions to "ignore embedded instructions" reduce risk but detect nothing, and logging or tracing only tells you afterwards.

Input sanitisation complements detection: strip or escape markup and tag-like strings from untrusted text, enforce length limits, and keep untrusted content clearly delimited from instructions. Least privilege for the tools an agent can call limits the damage when an injection does get through.

The most effective way to reduce hallucination about your own data is to ground the model in retrieved documents and show where each statement came from. Amazon Bedrock Knowledge Bases ingests documents from Amazon S3 (and other sources), creates embeddings, stores them in a vector store, and with RetrieveAndGenerate retrieves relevant chunks, generates the answer from them and returns citations linking statements to source documents. That is grounding plus a fact-checking trail with no embedding pipeline to manage. Fine-tuning does not ground answers, and stuffing whole documents into the prompt is costly and unreliable.

Three operational details matter:

• Custom prompt templates. RetrieveAndGenerate builds its generation prompt from a template. The $output_format_instructions$ placeholder injects the formatting instructions that produce citations; a custom template that drops it keeps retrieval working but returns answers with no citations. ($search_results$ is where retrieved chunks go.)
• Freshness. New or changed S3 objects are only picked up by an ingestion job (a data source sync, StartIngestionJob). Schedule it, for example with Amazon EventBridge Scheduler, rather than syncing by hand.
• Which versions are retrieved. If superseded documents must stay in the source, attach metadata through a .metadata.json sidecar file per document (for example a status attribute) and apply a metadata filter such as equals status current in the retrieval configuration.

Confidence scoring before generation. Retrieve returns a relevance score for each result. When many questions fall outside the corpus, calibrate a threshold on labelled answerable and unanswerable questions and abstain (hand off to a human, or say "I don't know") when the top score is below it. This saves the generation cost entirely, unlike checks that run on the generated answer. Asking the model to report its own confidence is not a reliable signal, and retrieving more passages does not create missing coverage.

Self-consistency when there is no reference. If answers come from the model's own knowledge, with no corpus or rule set to compare against, sample several answers at a non-zero temperature, embed them (for example with Amazon Titan Text Embeddings) and measure their semantic similarity. Answers that agree are more likely to be correct; divergent answers get flagged for review. Decoding at temperature 0 hides uncertainty instead of measuring it.

Guardrails offer two response-side checks for accuracy, and they answer different questions. A contextual grounding check scores whether a response is supported by the source text you supplied and whether it answers the user's query. An Automated Reasoning check validates a response against formal rules extracted from a policy document and explains which rules it satisfies or breaks.

Contextual grounding produces two scores, each with its own threshold from 0 to 0.99 (a threshold of 1 is not valid); a response scoring below a threshold is blocked:

• Grounding: is the response factually supported by the grounding source? Catches invented facts.
• Relevance: does the response address the query? Catches a response that faithfully restates the source but answers a different question than the one asked. Raising the grounding threshold cannot catch that.

The check runs on model responses and only when it has all three parts: a grounding source, a query and the response. With Converse or ApplyGuardrail you mark them with the grounding_source and query qualifiers on guardContent blocks; with InvokeModel you use the corresponding tags. Passages and questions sent as ordinary text produce no score, so no threshold change helps until they are qualified. On the Classic tier grounding is strict: any information not stated in the source, including reasonable inferences and friendly conversational phrasing, lowers the score. The Standard tier is designed for conversational and reasoning content (it needs cross-Region inference, and Standard-tier grounding is available in only some of the Regions that offer safeguard tiers). If correct answers are being blocked, move to Standard and lower the threshold, calibrating it on a labelled set of grounded and ungrounded answers so real hallucinations stay blocked.

Because Automated Reasoning checks are detect-only and do not support streaming APIs, an assistant using ConverseStream gets answers with no findings, and INVALID answers still reach users unless code acts on them. The fix is to generate the complete answer (Converse), read the findings in the guardrail trace, and correct or withhold anything not VALID before sending it. Neither check helps when there is no reference at all, which is where self-consistency (previous section) fits.

Two techniques remove whole classes of unreliable output: structured outputs force the response's shape, and text-to-SQL moves exact computation out of the model and into a database.

Structured outputs. For models that support it, Bedrock constrains generation to a JSON Schema so every response parses and contains the required fields. With Converse you set outputConfig.textFormat to a JSON schema; with InvokeModel it is output_config.format for Anthropic Claude models or response_format for open-weight models; for tool calls, strict tool use enforces the tool's input schema. This replaces prompt examples, temperature 0 and validate-and-retry loops, which reduce or detect malformed JSON but do not prevent it. Bedrock validates the schema first and returns a 400 ValidationException for unsupported features. Rules to know:

• Every object in the schema must set "additionalProperties": false.
• Numeric constraints (minimum, maximum, multipleOf) and string length constraints are not supported; check ranges in the code that reads the result. Enums of strings and arrays of strings are fine.
• The first request with a new schema is slower while the schema is compiled into a grammar, so allow for it in client timeouts. AWS notes the first compile can take up to about a minute and a half and recommends a generous client read timeout. Structured outputs also cannot be combined with Anthropic citations.

A schema guarantees format, not truth. When generated values must match data that changes (identifiers, balances, availability), generate them as schema-bound fields and then have a Lambda post-processing step check each one against the system of record before the response is sent. A schema can only constrain shape and fixed value lists; it knows nothing about the current state of a database.

Text-to-SQL for exact figures. A model totalling rows from retrieved text chunks will sometimes get the arithmetic wrong, however many chunks you retrieve. A structured knowledge base connects Bedrock Knowledge Bases to a structured store with Amazon Redshift as the query engine: it turns the question into SQL, the database computes the result, and the model writes the answer. The GenerateQuery API returns the SQL alone if you want to run it yourself. Consistency for business terms comes from the query generation configuration: table and column descriptions (for example, which column a business term such as "active customer" maps to), table inclusion or exclusion, and curated queries that pair example questions with the intended SQL.

Generated SQL is untrusted. Prompt instructions, input word filters and a prompt attack filter all try to persuade or detect; none constrains the SQL. Enforce safety outside the model: run the SQL as a database user with only SELECT on the approved views, and validate the statement before execution (a single SELECT that references only allowlisted objects).

Defense in depth means no single control is trusted to catch everything: each layer stops a different failure, so a bypass of one is caught by another. A typical GenAI request path has four layers:

1. API layer: AWS WAF (size, rate, IP reputation) and API Gateway request validation reject abuse and malformed requests before any compute runs.
2. Pre-processing: Lambda checks such as length, format, sanitisation and a classifier score (for example Amazon Comprehend) before the model is called.
3. Model-based guardrails: Bedrock Guardrails on input and output (content filters, denied topics, prompt attack, grounding).
4. Post-processing: Lambda validation of the response (schema, lookups against a system of record, business rules) before it is returned through the API.

Prefer the managed control at each layer when the scenario asks for least custom code: WAF instead of rate limiting in Lambda, a guardrail regex filter instead of a Lambda regex, request validators instead of validation code.

Choosing a safety classifier.

Amazon Comprehend prompt safety classification is closed to new customers (along with Comprehend topic modeling and event detection; accounts that used them recently keep access). AWS's migration path is a guardrail with the prompt attack filter called through ApplyGuardrail. Training a custom classifier for jailbreaks adds effort without matching the managed filter.

Safety controls need measurement before release and after every change. Amazon Bedrock evaluations score model outputs; ApplyGuardrail in a pipeline tests the guardrail itself.

Bedrock model evaluation job types:

Use the metric where the failure is a risk: Harmfulness on responses to red-team prompts, and Refusal on responses to legitimate prompts to measure over-blocking. To compare candidate models, run a judge job with your own prompt dataset against each model.

Evaluating responses produced elsewhere. An evaluation dataset can include each prompt's logged response in modelResponses (a response plus a modelIdentifier, one identifier per job). Bedrock then skips invoking a model and judges exactly those responses, so you can score a model on a SageMaker AI endpoint, or the final responses after a guardrail and post-processing, without moving the model or paying for inference again.

Automated adversarial testing of guardrail changes. Because ApplyGuardrail does not invoke a model, a CI/CD stage (for example in AWS CodeBuild) can run thousands of known attack prompts and a set of benign prompts against the guardrail's working DRAFT, compute the attack block rate and the false-block rate, and fail the build on any regression. Only a passing build creates a new immutable guardrail version for production.

Shadow trials on real traffic. To learn how many real requests a new policy would affect without changing anyone's experience, add it with the Detect (no action) action, enable the guardrail trace, analyse the assessments, then switch to Block. Detect mode is the only action that records what a policy would do while letting every request through unchanged.