AI-103 Set Up AI Solutions in Foundry: Deployment Types, Agent Setup and CI/CD

Current Foundry infrastructure is a Foundry resource (an Azure resource of kind AIServices) with one or more child Foundry projects. The resource is the unit of networking, resource-level RBAC, policy, keys and model deployments; a project is the unit of development work — its agents, files, evaluations and project-scoped access. No separate hub or Azure Machine Learning workspace is needed.

Hub-based projects still exist in Foundry (classic), built on an Azure Machine Learning hub, but new investment goes into Foundry projects. Treat a hub as the extra resource a new design avoids, not as a current choice for agent work.

An existing Azure OpenAI resource can be upgraded to a Foundry resource. The upgrade keeps the endpoint, API keys and existing deployments and state, and adds the project model, Foundry Agent Service and the wider model catalog. That is the low-change path when many apps already call the Azure OpenAI endpoint; creating a brand-new Foundry resource instead means a new endpoint and new keys for every caller.

Choosing the isolation boundary

Because model deployments and networking live on the resource, the boundary you pick decides what is shared:

Separate deployments inside one resource do give each workload its own tokens-per-minute limit, but network rules and resource-scoped role assignments remain shared, so they are not a full environment boundary.

The deployment type — not the resource's region — decides where prompts and responses are processed. The resource region governs data stored at rest; a Global deployment of a resource in an EU region may still process inferencing in any Azure region. Each type has a processing scope and a billing model:

Two rules decide most residency scenarios:

• Wider scope means higher default quota and more capacity. Global types carry the highest default quotas; Data Zone types sit between Global and the geography-based types. When a requirement is "EU-only" and also asks for the most quota, Data Zone Standard beats Standard.
• A data zone spans several countries. The EU data zone follows the EU Data Boundary, and other zones (for example in Asia Pacific) also cover multiple countries. A rule that processing stays inside one country's geography — Switzerland, Australia — needs a geography-based type: Standard for pay-per-token, Regional Provisioned for reserved capacity.

The Developer type (DeveloperTier) has no data residency guarantee at all, so treat it like a Global type when governing residency.

After residency, the workload's shape picks the billing model. Standard types are best-effort shared capacity billed per token; Provisioned types reserve throughput in provisioned throughput units (PTUs) billed per hour whether used or not; Batch types process files of requests asynchronously; Developer exists only to test fine-tuned models.

Adding another Standard deployment adds quota headroom but not latency consistency — both are still best-effort. And because a Developer deployment disappears after 24 hours, never point a staging or production app at one; redeploy the fine-tuned model as Standard or Global Standard instead.

Standard deployments of supported models can also run requests at different processing tiers, chosen with the service_tier request parameter (or, for priority, enabled on the whole deployment). The tier changes latency and price without changing the deployment type, so residency still comes from the type.

Priority processing is offered on Global Standard and on US Data Zone Standard deployments, so a US-only workload can have priority latency while keeping data-zone residency. It draws on the same quota as standard processing, and under ramp-rate limits or peak load a priority request can be served at standard processing and billed at the standard rate. Flex is narrower: it is available only on Global Standard deployments of supported models and shares that deployment's quota. Flex differs from Batch in the integration, not just the price: Batch means uploading an input file and polling an asynchronous job, while Flex keeps existing synchronous calls. Both tiers depend on the model being supported, so check the model list before designing around them.

Foundry deploys catalog models in one of two ways, and the model usually decides which. Serverless API deployments serve Foundry Models sold by Azure and selected partner models on Microsoft-managed shared capacity, billed per token, with the deployment types above. Managed compute (preview) hosts open-weight, partner and custom-weight models — for example the Hugging Face collection — on dedicated GPU virtual machines that Foundry sizes and patches, with a curated serving runtime, billed per hour for the compute.

When a model is offered both ways, Serverless API is the default choice: it brings built-in filters, data-zone options and provisioned capacity that managed compute lacks. Self-hosting on Azure Kubernetes Service or Container Apps can run any model, but it hands the cluster and serving runtime back to your team.

Every model deployment pins a model version, and its version upgrade option (versionUpgradeOption) decides what happens when Microsoft sets a new default version or retires the current one. The api-version query parameter is unrelated: it selects the API contract, not the model version.

Both automatic options can put an untested version into production before a regression suite runs. When a regulated workload must evaluate every change and avoid a retirement outage, the pattern is NoAutoUpgrade on production plus a planned migration: deploy the new version as a separate deployment, evaluate it, then switch the application before the retirement date.

A model deployment is a child resource of the Foundry resource — Microsoft.CognitiveServices/accounts/deployments — so it can be created with the Azure CLI, Bicep or ARM like any other resource. The deployment type is the deployment's SKU: sku.name (CLI --sku-name) takes values such as GlobalStandard, DataZoneStandard or GlobalProvisionedManaged, and sku.capacity (--sku-capacity) sets the quota or PTUs. The model's format, name and version, the versionUpgradeOption and the content filter (raiPolicyName) are deployment properties.

az cognitiveservices account deployment create \
  -n $accountName -g $resourceGroupName \
  --deployment-name chat-model \
  --model-name gpt-4.1 --model-version 2025-04-14 \
  --model-format OpenAI \
  --sku-name DataZoneStandard --sku-capacity 50

Do not confuse the deployment SKU with the account's --kind (for example AIServices), which is set when the Foundry resource itself is created.

Governing which types can be created

Because the type is a property of the deployment resource, Azure Policy can evaluate the deployment's sku.name and deny the ones you do not allow. To keep all inferencing inside the EU, deny the Global SKUs (GlobalStandard, GlobalProvisionedManaged, GlobalBatch) and DeveloperTier across the management group or subscriptions. A policy on resource location cannot do this — every resource may already sit in an EU region — and an RBAC role that removes deployment rights blocks compliant types too.

The agent setup decides where Foundry Agent Service stores agent state. Basic setup keeps conversations, files and vector stores in Microsoft-managed resources — quickest to start. Standard setup brings your own Azure resources in your own tenant, so all agent data stays under your control:

A customer-managed key encrypts data but does not move it into resources you own, and diagnostic logs copy telemetry rather than the agent's state — neither substitutes for standard setup when ownership is the requirement.

Capability settings are fixed at project creation

The links from a project to its Cosmos DB, Storage and AI Search resources are capability settings. They can be set on the account (as defaults for new projects) or on the project, but they cannot be updated on an existing project. Moving a basic-setup project to your own stores therefore means creating a new project with the right capability settings and recreating the agents there; redeploying the template or adding ordinary project connections does not change where agent state lives.

Standard setup also isolates data per project: each project gets its own Cosmos DB containers with their own throughput, so the Cosmos DB account's total throughput must grow with every project. A throughput error while provisioning a new project points at the account's RU/s, while a missing role assignment shows up as authorization failures when agents run.

A prompt agent is declarative: a model deployment, instructions and tools configured in the portal or SDK, which Foundry runs for you with no code or containers to own. A hosted agent is your own agent code — LangGraph, Microsoft Agent Framework or custom orchestration — packaged as a container image in Azure Container Registry and run by Foundry with a managed endpoint, per-session scaling, a Microsoft Entra identity and built-in tracing. Choose a prompt agent when instructions plus tools are enough; choose a hosted agent when you have custom orchestration code but still want Foundry's managed runtime instead of operating Container Apps or AKS yourself. (Managed compute is not an agent runtime — it hosts model weights.)

Hosted agent versions

• Each version is an immutable snapshot of the container image, CPU and memory, environment variables and protocol configuration. Changing any of them — a new model deployment name in an environment variable, a bigger size — means creating a new version.
• An agent endpoint sends all traffic to one version. There is no percentage traffic split; rolling back means deploying the previous version again.
• Hosted agents scale per session: each session gets its own VM-isolated sandbox of the version's size (sizes such as 0.5 vCPU/1 GiB up to 2 vCPU/4 GiB), released after an idle timeout. There are no replica counts or warm pools, and cost follows CPU and memory across active sessions — oversizing multiplies with concurrency.

Protocols

Identity

Each hosted agent gets a dedicated Microsoft Entra agent identity that its container uses at runtime for models, tools and downstream Azure services. To let one agent read your storage account without keys, assign the data role (for example Storage Blob Data Reader) to that agent's identity. The project managed identity is used by the platform for infrastructure tasks such as pulling the image from the registry and is shared across the project, so granting it access widens the scope to every agent. Never put connection strings, keys or SAS tokens in the image or a version's environment variables.

CI/CD for Foundry means treating everything as code — infrastructure, model deployments and agent definitions — and promoting it through separate dev, test and prod environments with keyless authentication and an evaluation gate.

Agents as code

Store agent definitions in the repository, review them by pull request, and let the pipeline create them in each environment's project. In the azure-ai-projects 2.x SDK, project.agents.create_version(agent_name=..., definition=PromptAgentDefinition(model=..., instructions=...)) creates the agent, or a new immutable version if the name already exists, giving the evaluation step a name:version to reference. create_agent and update_agent belong to the classic assistants-style API, where agents were edited in place; get_openai_client returns a client for calling agents and models.

Keyless pipeline authentication

Pipelines should hold no long-lived Azure secrets. With workload identity federation (OpenID Connect), the pipeline exchanges a short-lived token from its own identity provider for a Microsoft Entra access token:

• GitHub Actions: grant the job permissions: id-token: write so GitHub issues an OIDC token, add a federated credential on the app registration (or user-assigned managed identity) whose subject matches the repository and branch or environment, and call azure/login with the client, tenant and subscription IDs — no client secret.
• Azure DevOps: use an Azure Resource Manager service connection with workload identity federation.

Client secrets, Foundry API keys and personal access tokens stored as pipeline secrets are exactly what this avoids; an API key also only authorizes data-plane calls and cannot deploy Bicep. Give the pipeline identity least-privilege roles for what it does: a control-plane role on the resource to create deployments, and the Foundry User role (formerly Azure AI User) on the project to create agents.

Evaluation gates

Both invoke the agents with the dataset, score the answers with Foundry evaluators and publish a summary in the run. Agents are listed as comma-separated agent-name:version values; when several are listed the report compares them with statistical significance against a baseline — the agent named in baseline-agent-id, or the first agent listed if that input is omitted. List the production version first (or name it as the baseline) so the new version is judged against it, and fail the pipeline when scores regress.

A sound promotion design combines these: definitions in source control, a new version created by the pipeline in each environment's project, and an evaluation gate before prod. Editing agents directly in the prod portal, merging environments into one project, or tightening content filters (which detect harmful categories, not answer quality) do not provide review or quality gating.