AI-103 Responsible AI in Foundry: Guardrails, Safety Evaluations, Auditing and Agent Governance

A guardrail in Microsoft Foundry is a named collection of controls that screens the traffic of a model deployment or an agent for harmful or risky content. Each control combines three things: a risk to detect (for example Violence, user prompt attacks or personally identifiable information), the intervention points where it scans, and the action to take when the risk is found. In Azure Resource Manager a guardrail is an RAI policy, which is why its property and parameter names say rai.

Every deployment starts with a Microsoft-managed default guardrail, Microsoft.DefaultV2 (the older default is Microsoft.Default). Default guardrails can't be edited. To change behaviour you create a custom guardrail and assign it. Inside a custom guardrail the core harm controls (hate, sexual, self-harm and violence) can be overridden with different settings, but they can't be deleted.

The risks a guardrail can detect

• Harm categories — hate, sexual, self-harm and violence, each with a blocking level.
• Prompt Shields — user prompt attacks (jailbreaks typed by the user) and document attacks, also called indirect attacks or indirect prompt injection (instructions hidden in retrieved documents or tool results).
• Protected material — known copyrighted text, and code that matches public GitHub repositories.
• Personally identifiable information (PII) (preview) — entity types such as Email, PhoneNumber or Credit Card Number in model output.
• Groundedness (preview) and Task adherence (preview) — signals about whether output is supported by its sources and whether an agent's actions match what the user asked.
• Custom blocklists — exact terms you want blocked. A blocklist catches only the phrases you list, so it is no substitute for attack detection.

The actions

Annotate reports what was detected in the response metadata and lets the content through. Annotate and block reports it and stops the content: the request or output is blocked, and at a tool call point the call isn't executed. Annotations are how an app or analyst sees severity results without filtering.

An intervention point is the place in the request flow where a control scans content. Model deployments have two points, and agents have four:

Choosing the point is about when the risky content exists. Data an agent passes as arguments to an external system (for example the text of a ticket it files or a message it sends) exists only in the tool call, so a control on the output point never sees it if the chat reply itself is clean. Hidden instructions in a web page a tool fetches exist in the tool response, and scanning the output is too late because the agent has already acted on them. User prompt attack detection looks only at what the user typed.

For model calls, document attacks are scanned at user input, inside documents that the app delimits (see Prompt Shields below). For agents, they can also be scanned at the tool response point.

Only moderation-supported tools are screened

Tool call and tool response controls take effect only for tools that support moderation: currently Azure AI Search, Azure Functions, OpenAPI, SharePoint Grounding, Fabric Data Agent, Bing Grounding, Bing Custom Search and Browser Automation. For other tools — notably the MCP tool — the controls can be configured but do nothing. So when a tool's traffic must be screened, the rule is to connect it through a moderation-supported tool type; an MCP tool's calls and responses pass the tool controls unchecked. Check the current list on Microsoft Learn, because it grows.

The four harm categories use a blocking level per intervention point, and the level names describe how much the control blocks — not which severity it targets:

A common mistake is to read High as "block high-severity content only"; it is the strictest setting. Work it out from what must still get through: to let low- and medium-severity content in a category pass while stopping only the most severe, you choose Low; to let only low severity pass, Medium.

Turning blocking down needs approval

Setting a harm category on outputs to annotate-only or Off is a modified guardrail, and it requires the subscription to be approved through Microsoft's Limited Access review for modified guardrails. Until that approval exists, outputs keep blocking. Editing the default guardrail, deleting a core control or picking a different guardrail per request doesn't bypass it.

PII and protected material

The PII control (preview) runs on output, and its mode is set per entity category: Annotate flags the data, Annotate and block blocks the whole output when that category appears. You can therefore block credit card numbers while only flagging email addresses.

Protected material has two separate controls. The text control detects known copyrighted text such as lyrics or articles. The code control detects code that matches public GitHub repositories, and its annotation returns a citation URL and the repository's license. With Annotate, the suggestion stays visible and the app can display the citation and license, which Microsoft recommends (respecting the license can matter for Customer Copyright Commitment coverage). Annotate and block hides the code instead.

Prompt Shields are the guardrail risks that detect attempts to subvert the model's instructions. They separate two attack types by where the attack text comes from:

Delimit documents or they look like user input

With the Chat Completions API, the guardrail knows which text is a document only if the app wraps it in the documents delimiter: """ <documents> ... </documents> """, with the content JSON-escaped. Pasting retrieved content straight into the user message makes it indistinguishable from what the user typed, so document attack detection never runs on it (and groundedness checks have no sources). Don't move untrusted content into the system message to "hide" it — that gives it more trust, not less.

Spotlighting (preview)

Spotlighting is an optional toggle on the document attack control. It base-64 encodes the delimited documents so the model treats them as less trustworthy than the system and user prompts. Know its limits:

• It is off by default and supplements document attack detection; it doesn't replace it.
• It works only for models called through the Chat Completions API, and it doesn't apply to agents.
• Encoding increases token count, which raises cost and can push large documents past input limits; replies may occasionally mention the encoding.

A guardrail is assigned at one of three levels, and the most specific assignment wins for the traffic it covers.

Model deployments

A deployment uses the guardrail named in its raiPolicyName property. Setting it in a Bicep or ARM template (or the portal) applies the guardrail to every caller of that deployment without touching their code, and keeps the change in your CI/CD pipeline.

Per request

An app can name a custom guardrail in the x-policy-id header of an individual call. It overrides the deployment's guardrail for that call only — the two aren't merged — so one deployment can serve tenants with different policies. Limits: image input scenarios don't support it and use the default guardrail, and a name that doesn't exist returns an InvalidContentFilterPolicy error rather than falling back. Because it relies on each app sending the header, it can't enforce a policy on callers you don't control.

Agents

• An agent with no guardrail assigned inherits its model deployment's guardrail.
• An agent with its own guardrail uses only that one: it fully overrides the deployment guardrail. Editing the deployment's guardrail then has no effect on the agent, and removing the agent's guardrail would also drop its other controls.
• One guardrail can be assigned to both a deployment and an agent. Controls whose risk isn't supported for agents — Spotlighting and Groundedness — are skipped for the agent but still apply to the model. PII, task adherence and indirect attacks do apply to agents.
• For agents, controls use the Annotate and block action.

Hosted agents (your own agent code running in Foundry Agent Service) get content safety through the rai_config object in the agent definition. Its rai_policy_name takes the guardrail's full ARM resource ID, not its bare name.

A hosted agent's own endpoint is screened only by its rai_config; a guardrail on a model deployment it calls applies to those model calls, not to the agent's endpoint. If a referenced policy is missing, screening fails open, so verify the ID.

Protocol matters

On the responses protocol the platform knows the request and response shapes, so the policy ID is enough. On the invocations protocol bodies are custom, so you must also add invocations_moderation: input_paths selecting the user text, output_paths selecting text in buffered JSON replies, stream_selectors for server-sent events, and a response_mode of streaming, buffered, or both. Without it the policy screens nothing and requests still return HTTP 200. A successful reply in a shape the response_mode doesn't declare is rejected with HTTP 502 rather than passed unscreened, so the declared mode must cover every response shape the agent actually returns, together with the output paths to scan. The invocations_ws (WebSocket) protocol has no content safety moderation.

Network egress controls (preview)

The hosted agent's RAI policy can also hold egress rules, enforced inside the agent's sandbox, so outbound traffic changes without code changes. Rules are evaluated top to bottom, the first match wins, and unmatched requests get the default action.

• Allow sends the request to its host; Deny blocks it.
• Transform edits headers on the original request (for example adds a workload header).
• Rewrite redirects the request to a different destination — the way to route a partner host through an API management gateway.

A default of Deny plus Allow rules is an allow list, the recommended design; a default of Allow is only a deny list. Roll out in Audit mode first: traffic flows, requests that would be denied are logged as would-deny decisions, and Transform and Rewrite still apply. When the logs show every needed host, switch to Enforce, where denied calls get HTTP 403. Egress controls fail closed and are available for hosted agents only.

Groundedness detection and Task Adherence are runtime risk signals: they don't measure harm, they check whether output or an action is justified.

Groundedness detection

Azure AI Content Safety groundedness detection compares generated text with the grounding sources you supply and flags content those sources don't support. You tune it with:

• Domain — Medical (healthcare and scientific content) or Generic (everything else, such as banking or retail support).
• Task — Summarization or QnA.
• Mode — Non-Reasoning is fast and returns only grounded or ungrounded, for latency-sensitive production paths; Reasoning returns explanations for each ungrounded segment, for development and debugging.
• Correction (preview) — returns corrected text that replaces unsupported details (for example a wrong name or dosage) with values from the sources.

Domain and task change detection accuracy, not whether explanations are returned. The groundedness guardrail control is preview and applies to models only. An offline groundedness evaluator scores stored data in bulk; it can't fix each response in line.

Task Adherence (preview)

Task Adherence analyses the tools available, the conversation and the agent's planned tool calls, and flags calls that don't match what the user asked — such as a planned call that changes or submits something when the user only asked to look something up. Its response has two fields: taskRiskDetected (Boolean) and details (the reasoning when a risk is detected). Microsoft recommends blocking or escalating to a person when it's true. It can run as a guardrail control for agents or be called directly.

This makes it the efficient form of human oversight: instead of a reviewer approving every sensitive call, aligned calls run automatically and only flagged ones go to a reviewer, with the details text as the explanation.

Responsible AI instrumentation measures safety before and after release, alongside the runtime guardrails. Foundry offers two tools for it: risk and safety evaluators, which score responses you already have, and the AI Red Teaming Agent, which generates attacks to probe a model or agent.

Risk and safety evaluators

These evaluators run on Microsoft-hosted safety models, so unlike LLM-as-judge quality evaluators they don't need a deployment_name. In a cloud evaluation they are referenced as builtin.<risk>:

The four harm evaluators return a severity score from 0 to 7, a pass/fail label and a reason. With the default threshold of 3, scores 0–3 pass and 4–7 fail. A run reports the defect rate: the share of responses with undesired content.

AI Red Teaming Agent

The AI Red Teaming Agent automates adversarial testing. It combines PyRIT attack strategies — such as Base64, Flip (character flipping), Leetspeak, Crescendo and Jailbreak — with the risk and safety evaluators, scores each attack–response pair, and reports the Attack Success Rate (ASR), the percentage of attacks that got harmful output. Use it from design time through pre-deployment and after release.

• A local scan (the RedTeam class in azure-ai-evaluation) covers the standard harm categories.
• Cloud red teaming adds the agentic risk categories — prohibited actions, sensitive data leakage and task adherence — runs agents in a minimally sandboxed environment, and redacts the generated harmful inputs from results so reviewers aren't exposed to them. It runs only in specific regions (Sweden Central is one), so the project must be in a supported region; check the current list.

Explanation tooling

"Explanation" in this skill means the signals that say why something was flagged: the reason field each evaluator returns, groundedness Reasoning mode, the Task Adherence details field, and the red-teaming scorecard broken down by risk category and attack strategy.

An audit trail for an AI system records what the model and agent did, who approved risky steps, and where generated content came from. In Foundry that comes from three sources: traces, approval records and Content Credentials.

Trace logging

Tracing sends OpenTelemetry spans to the Application Insights resource connected to the Foundry project. Spans capture model calls, tool invocations, agent decision flow and latency — the record an auditor needs. Content recording is a separate switch that also stores user inputs, model outputs and tool arguments and results. Microsoft's guidance is to turn content recording on in development and off in production when the content is sensitive. Shorter retention or tighter RBAC limit exposure, but the content is still stored; turning tracing off loses the trail.

For people who only need to read traces, start with Log Analytics Reader on the workspace, and add Privileged Monitoring Data Reader when the trace tables are protected tables. Contributor roles allow changes or ingestion and break least privilege. Log tool calls and approval decisions too, so oversight is itself auditable.

Provenance metadata: Content Credentials

Every image generated by the DALL-E and GPT-image-1 series models in Azure OpenAI carries Content Credentials automatically — there's no setting to enable or form to request. They follow the C2PA standard: a manifest cryptographically signed by a certificate that traces back to Azure OpenAI, containing the description "AI Generated Image", the softwareAgent (for example Azure OpenAI ImageGen) and the creation time. EXIF fields are not that signed manifest. To verify:

• a person uses the Content Credentials Verify website;
• an automated pipeline uses the Content Authenticity Initiative (CAI) open-source tools, which validate manifests in code.

Detecting unsafe images, watermarks and visual policy rules belongs to responsible AI for multimodal content in Domain 2.

Governing agent behaviour means limiting which tools an agent can call, who must approve risky calls, and how credentials and tool output are handled. These are separate controls, and each fails if used for another's job.

Approval workflows (oversight modes)

The MCP tool's require_approval defaults to "always" and accepts four forms:

• "always" — every call needs approval;
• "never" — no call needs approval (no oversight);
• {"never": [...]} — listed tools skip approval, every other tool needs it;
• {"always": [...]} — listed tools need approval, every other tool runs freely.

The two dictionary forms behave the same for today's tools but differ for tools a server adds later: a never list keeps new tools behind approval, while an always list lets them through.

When approval is needed, the response output contains an mcp_approval_request item with the server label, tool name and arguments. The app answers with an McpApprovalResponse per request — approve true or false and approval_request_id set to the request's ID — in a new responses.create call whose previous_response_id is the earlier response's ID, so the same run continues:

inputs = [McpApprovalResponse(
    type="mcp_approval_response",
    approve=True,
    approval_request_id=item.id)
    for item in response.output
    if item.type == "mcp_approval_request"]
response = openai.responses.create(
    input=inputs,
    previous_response_id=response.id,
    extra_body={"agent_reference": {
        "name": agent.name, "type": "agent_reference"}})

Credentials and untrusted tools

Store an MCP server's key in a project connection and reference it with project_connection_id; that keeps it out of the agent definition and app code. Custom headers put the key in the definition, server_label is only a name, and a managed identity helps only when the server accepts Microsoft Entra tokens.

Microsoft's MCP guidance: allow-list tools, treat tool descriptions and results as untrusted input (they can carry indirect prompt injection), require approval for high-risk write operations, review tool names and arguments before approving, use servers hosted by the provider rather than rehosting proxies, and log approvals and tool calls.