AI-103: Responsible AI for Images — Content Safety, Prompt Shields and Visual Policy

Responsible AI for multimodal content means checking images — the ones users upload and the ones your models generate — for harm, hidden instructions and policy breaches before they can do damage. AI-103 task 3.3 has three skills, and each maps to a different set of tools:

A useful rule of thumb runs through the whole topic: the AI services produce signals (severities, attack flags, field values), and the application or the guardrail makes the decision. Knowing which service produces which signal, and where the decision is made, answers most design questions here.

Image generation controls and Content Understanding analyzer mechanics belong to the sibling tasks (3.1 and 3.2), and agent-wide guardrail governance belongs to Domain 1; this lesson uses them only as they apply to visual safety and policy.

You classify an unsafe image by sending it to the Azure AI Content Safety Analyze Image API, which returns a severity for each of four harm categories — Hate, Sexual, Violence and Self-harm — and then comparing each severity with a threshold your application chooses. The API scores; it does not block. That makes it the right tool when no generative model is in the flow, such as moderating user uploads before they are published.

The image severity scale

Text analysis can return the full 0–7 scale, but the current image model returns a trimmed scale of four values: 0, 2, 4 and 6. Each pair of adjacent levels collapses into one value:

This matters when you write thresholds. Odd values such as 3 or 5 never occur for images, so a threshold written for the text scale can quietly change meaning: "reject at 3 or above" behaves exactly like "reject at 4 or above" for images, and low content (2) is allowed. Write image rules against the values that actually occur — 2, 4 and 6 — so the rule says what it does.

Calling it from Python

With the azure-ai-contentsafety package, you wrap the image in ImageData inside AnalyzeImageOptions. ImageData takes either content (the raw image bytes) or blob_url (an Azure Blob Storage URL the service reads itself):

from azure.ai.contentsafety import ContentSafetyClient
from azure.ai.contentsafety.models import AnalyzeImageOptions, ImageData

client = ContentSafetyClient(endpoint, credential)
with open(path, "rb") as f:
    request = AnalyzeImageOptions(image=ImageData(content=f.read()))
response = client.analyze_image(request)

for item in response.categories_analysis:
    print(item.category, item.severity)

Pass the raw bytes: the SDK base64-encodes them for the REST body, so encoding them yourself first corrupts the payload. A file name or local path is not something the service can fetch. In REST, the same request carries image.content as base64 or image.blobUrl, and the image response uses outputType FourSeverityLevels.

The Analyze Image API accepts images up to 4 MB and between 50 x 50 and 7,200 x 7,200 pixels, in JPEG, PNG, GIF, BMP, TIFF or WEBP format. Requests outside those limits are rejected, so camera originals (often far larger in both bytes and pixels) must be downscaled and compressed before analysis. The limits apply to the image itself, whichever way you deliver it, so the fix is always to reduce the image: fewer pixels and lossy compression. Changing only the container or delivery method leaves an oversized image oversized.

Animated images: only the first frame

For an animated format such as an animated GIF, the service extracts the first frame and analyzes only that. A sticker or animation can therefore open with harmless frames and show harmful content later and still pass. If animations are a risk, split them in your application and send frames (every frame, or a sampling dense enough for your risk) as separate images, then combine the results — for example, take the highest severity per category.

Reading images from Blob Storage without keys

When you pass a blobUrl, the Content Safety service downloads the image itself, so it is the Content Safety resource — not your app — that needs access. To avoid SAS tokens and account keys, Microsoft documents this setup:

1. Enable the Content Safety resource's system-assigned managed identity.
2. Assign that identity Storage Blob Data Contributor or Storage Blob Data Owner on the storage account.

Microsoft states these are the only valid roles for this scenario — Storage Blob Data Reader looks like the least-privilege choice but is not listed, and the control-plane Reader role grants no access to blob data at all. Granting a role to your application's identity does nothing here, because the app never makes the read.

You moderate an image and its text together with the Content Safety multimodal (image-with-text) API, which analyzes an image plus associated text in one request and returns harm severities for the combination. Set enableOcr to true and the service also runs OCR on the image and analyzes the recognised text along with the visual content. This catches the classic meme problem: a harmless photo with a hateful caption, which Analyze Image scores as safe because it judges only the visual content.

The multimodal API has tight text limits worth remembering:

• With enableOcr, it analyzes at most 1,000 characters of text recognised in the image; anything beyond is truncated.
• Its optional text field is also limited to 1,000 characters.

Text-heavy images — long chat screenshots, scanned pages — can therefore hide harmful text past the first 1,000 characters. Because both text inputs share the same cap, the multimodal API alone can't cover long text. The simple fix is to keep the multimodal call for the visuals and also score the full extracted text with the Analyze Text API, which accepts up to 10,000 characters per request (split longer text into several requests).

Foundry guardrails filter images by applying content-harm controls (Hate, Sexual, Violence, Self-harm) to the inputs and outputs of a model deployment or agent, including image inputs to vision-capable models and images produced by image-generation models such as the GPT-image-1 series. Unlike the standalone Content Safety APIs, a guardrail enforces the decision for you: assign it to the deployment and blocked content never reaches the model or the caller, with no moderation code in the app.

Intervention points

A guardrail control applies at one or more intervention points: user input, tool call, tool response and output. Model deployments use user input and output; tool call and tool response apply to agents only. Images attached to a prompt are part of the user input, so a control at user input stops them before the model processes them; the call fails with an HTTP 400 error carrying the content_filter code, which the app can catch. A control at output filters what the model returns — for an image model, the generated image.

Blocking levels

Each control has a blocking level, and the names read backwards from what many people expect: the level names how sensitive the filter is, not which severity it blocks.

Content classified as safe is never blocked and isn't configurable. So "block anything above safe" means the High blocking level, applied at every point that must be covered — for a generation app, both user input (the prompt) and output (the image). To loosen filtering for legitimate use, such as nudity in classical art, move the control toward Low, which still blocks high-severity content.

Turning a control off

Turning a content-harm control off completely is allowed on inputs for all customers. Turning it off on outputs for models sold directly by Azure requires approval through the Limited Access Review for modified guardrails.

You detect indirect prompt injection carried in an image by extracting the image's text first (for example with OCR or Azure Content Understanding) and sending that text to Prompt Shields as a document. Prompt Shields analyses supplied text; it is not documented to read pixels, so the extraction step is what makes image-borne instructions visible to it.

Prompt Shields distinguishes two attack types by where the text comes from:

Text from a receipt photo or screenshot is third-party content, so it belongs in documents. The REST call is POST {endpoint}/contentsafety/text:shieldPrompt?api-version=2024-09-01:

pages = [ocr_text_page1, ocr_text_page2]
payload = {"userPrompt": user_message, "documents": pages}
resp = requests.post(shield_url, headers=auth_headers, json=payload)
verdicts = resp.json()

flagged = [
    page for page, doc in zip(pages, verdicts["documentsAnalysis"])
    if doc["attackDetected"]
]
prompt_attack = verdicts["userPromptAnalysis"]["attackDetected"]

Here shield_url is the text:shieldPrompt URL above. Because results come back in the order the documents were sent, zipping the two lists tells you exactly which page carried the attack.

The response holds Booleans, not severities — severities and categoriesAnalysis belong to the harm APIs. Harm APIs are also the wrong check for injection: a quiet "approve this claim" instruction contains nothing hateful or violent.

Request limits

One Prompt Shields request accepts a user prompt of up to 10,000 characters and up to five documents with a combined 10,000 characters. Multi-page scans must be batched so that every request satisfies both limits at once — the document count and the combined character total — so whichever limit a batch reaches first decides its size. For example, short pages of about 1,500 characters fit five to a request (the count limit binds first), while longer pages hit the character limit before the count. The whole upload counts as attacked if any document returns attackDetected.

They don't catch it because both act on delimited document text, and a photo sent only as an image_url content part isn't document text. In a guardrail, document attack detection for model calls inspects document content at the user input point, and Microsoft notes that detecting indirect attacks requires the prompt to mark documents with document delimiters. Spotlighting — a Prompt Shields option that transforms document content (by base64-encoding it) so the model treats it as data rather than instructions — likewise works on that document content, and it is available for Chat Completions, not for agents.

So a Chat Completions app that sends a contract photo purely as an image part gets no injection protection from either control, however high it sets them. The fix is to extract the photo's text and send it as a delimited document, which brings it under both document attack detection and Spotlighting. Raising the vision detail setting doesn't help and can make the printed text easier for the model to read.

You limit the impact of image-borne injection with defence in depth: assume some injected text will get past detection, and design so that it can't achieve much. The controls that work:

• A tool-less extraction step. Have a step with no tools convert each image into fixed schema fields (booking reference, amount, date). The agent that holds tools then receives typed data instead of the image's free text, so there is nowhere for an instruction to hide.
• Human approval for consequential tools. Require a person to approve actions such as changing reservations or approving payments, so a manipulated plan is stopped before it takes effect.
• Detection before the agent runs. Check extracted text with Prompt Shields as a document (previous sections).
• Least privilege for tools. Give the agent only the tools and scopes the task needs.

Generation and sampling settings shape how a model phrases its output, not whose instructions it follows, so tuning them is not an injection defence. The effective controls change what the model sees or what its actions can do.

The app decides what to do with a detection

Prompt Shields returns verdicts; it doesn't block or route anything. Your application chooses the response, and it should be proportionate to where the attack was found. If documentsAnalysis flags the screenshot text but userPromptAnalysis is clean, withhold that document from the model (and flag it for review) while still answering the user's legitimate question. The general principles: scope the response to the content that was flagged rather than to the whole conversation, and never rely on telling the model to ignore content you have already identified as an attack — if flagged text still reaches the model, the attack still has its chance. A document verdict carries as much weight as a user-prompt verdict.

You meet a visible-labelling requirement by having the application composite a visible label or watermark into the image's pixels after generation and before publishing. Two principles decide this: a requirement that must hold every time belongs in deterministic code rather than in a generation instruction, and a label people must see has to be in the pixels — anything stored beside the image (storage metadata, tags, database fields) travels separately and is invisible to a viewer.

Separately, Azure OpenAI attaches Content Credentials to every image generated by the DALL-E and GPT-image-1 series models. Know how they differ from a visible watermark:

A provenance policy often uses both: Content Credentials for machine-verifiable origin, and a visible label where regulations or brand policy require anyone to see it. Note that editing or re-encoding an image can strip its metadata, so publishing pipelines should preserve the manifest where it matters.

You flag known harmful symbols with the built-in classifiers, flag organisation-specific or newly emerging symbols with custom categories (rapid) image incidents, and check brand usage with Azure Content Understanding classify fields — then combine those signals with deterministic rules in code.

Built-in categories vs custom incidents

Widely known hate symbols (such as glorified Nazi imagery) fall within the Hate category's image definitions, so Analyze Image with your own threshold handles them. A brand-new emblem or a local gang logo is not something the built-in classifier learned, and lowering the Hate threshold won't teach it. For that, the custom categories (rapid) API is designed for emerging incidents, with no training step:

1. Create an image incident (PATCH /contentsafety/image/incidents/{name}).
2. Add example images with :addIncidentSamples (up to 1,000 per incident).
3. Deploy it with :deploy — an incident isn't available for analysis until it is deployed.
4. Call image:detectIncidents with the incident names to check new images against the samples.

Brand usage with Content Understanding

Brand checks ask fixed questions about every image — is the logo visible, is it the approved version, is a competitor's sign in view. Build a Content Understanding analyzer whose baseAnalyzerId is prebuilt-image and define one field per question using the classify method, which picks a value from a fixed enum (Yes/No, Approved/Outdated/Missing). Every image returns the same shape, which a rules engine can act on directly. The principle: when downstream code acts on the answer, prefer a field whose possible values are fixed in advance over one whose output is open-ended.

Decide in code

When a policy combines several rules and auditors need consistent, explainable outcomes, gather one signal per rule — a Content Understanding field, a rapid incident match, an Analyze Image severity (for example, Sexual no higher than low, which is 2) — and make the final decision in deterministic code. The same signals then always give the same decision, and each rejection names the rule that failed. Keep models and classifiers as signal producers and keep the policy decision itself in code you can test and audit; guardrails, for their part, cover harm categories rather than brand rules.