AI-103: Image and Video Generation and Editing in Foundry — GPT-image, FLUX and Sora 2

Microsoft Foundry generates images with two model families: the GPT-image series from Azure OpenAI and Black Forest Labs' FLUX models. You deploy one into your Foundry resource and call the deployment by name. The choice usually comes down to three things: whether the model is generally available or needs a limited-access approval, which output sizes it accepts, and which API shape your code already uses.

dall-e-3 is retired. Microsoft retired it on 4 March 2026: it can't be deployed, and existing deployments no longer work. The replacement is a GPT-image series model, which uses the same images API, so migrating means creating a new deployment and pointing the calls at it. Redeploying a different dall-e-3 version or changing its deployment type does not bring it back. The one code change to expect is in the response: GPT-image models return base64 data rather than a URL (next section).

Microsoft also lists newer GA variants (GPT-image-2.5 Sunburst and Flare) with extra quality levels. The selection logic is the same: check availability, size support and the API before you commit.

You generate an image by calling images.generate on an image deployment, passing the deployment name as model and a text prompt. With the OpenAI Python client pointed at the resource's /openai/v1/ endpoint, the call is the same as on OpenAI's own API.

import base64
result = client.images.generate(
    model="gpt-image-1",          # your deployment name
    prompt="Flat-lay photo of a leather notebook",
    size="1536x1024",
    quality="medium",
    n=1,
)
image_bytes = base64.b64decode(result.data[0].b64_json)
with open("notebook.png", "wb") as f:
    f.write(image_bytes)

Three response rules matter:

• GPT-image models always return base64. Each item in result.data carries b64_json; there is no URL. Code written for a URL-returning model reads data[0].url, gets None, and has to decode b64_json and write the bytes instead.
• response_format is not supported for GPT-image models, so you can't ask for a URL. output_format is a different setting: it picks the image encoding (PNG or JPEG), not how the image is delivered.
• n sets how many images one call returns: 1 to 10, default 1. It multiplies the work, so it doesn't speed anything up.

A text prompt alone can't reproduce a specific existing object, such as one particular house or product. When the output has to start from a real image, use an edit request (or a reference-image model) instead of generation.

The GPT-image models expose a small set of controls on images.generate. Each one changes a single thing, and most scenario questions come down to picking the one that changes the thing the requirement names.

Transparent backgrounds

For stickers, icons or cut-outs, set background="transparent" and output_format="png". PNG is the only supported output format with an alpha channel, so transparent plus JPEG can't work. auto leaves the decision to the model and opaque asks for a filled background. Microsoft documents transparency for gpt-image-1 and gpt-image-2; confirm support before relying on it with another model.

Faster drafts versus faster-feeling pages

Two different problems are easy to confuse. If generation itself is too slow (for example, a storyboard tool that renders dozens of rough thumbnails before anyone picks one), lower quality for the drafts and keep high for the final image; the model and size stay the same. If the wait is acceptable but users see a blank page, stream the request: with stream=True and partial_images set, the API emits image_generation.partial_image events, each with b64_json data the page can show as a rough preview that sharpens before the final image arrives. Streaming improves perceived latency but doesn't finish sooner, and compression only shrinks a JPEG file.

You edit an image by calling images.edit (the /images/edits endpoint) with one or more input images and a prompt, and optionally a mask that limits where the change may happen. The request format differs from generation:

• multipart/form-data, not JSON. Images and the mask are uploaded as file parts. Embedding base64 images in a JSON body returns errors.
• Up to 16 input images per request, each a PNG or JPG under 50 MB. One request can combine several images, for example a set of furniture photos composed into a single room, so you don't need to split the work. WebP inputs need converting first.
• The generations endpoint takes no input images. Editing always goes through the edits endpoint.

Prompt-driven edits (no mask)

Without a mask, the model starts from your image and applies the prompt wherever it's relevant. That suits scene-wide changes that keep the subject recognisable, such as restyling a city street photo as a watercolour, turning daytime into dusk, or replacing a background while keeping the composition.

Mask-based edits (inpainting)

Inpainting changes only one region and leaves every other pixel as it is. The mask follows three rules, and all three must hold:

A fully opaque mask therefore allows no change at all. When a mask is invalid, correct the mask itself rather than altering the source image.

Preserving faces and features: input_fidelity

input_fidelity controls how hard the model works to match the style and features of the input images, especially faces. Setting it to high preserves faces more accurately, which matters for catalogue or portrait edits where a person must stay recognisably the same. It is not supported by gpt-image-1-mini, so a workload on the mini model needs another GPT-image deployment (such as gpt-image-1) to use it. Other settings don't do its job: quality adds detail rather than matching the input, and input_fidelity also doesn't confine an edit to a region; only a mask does that.

Use a FLUX model when its strengths fit the job: in-context editing with consistent characters, multi-reference editing, or fast text-to-image. Each FLUX model plays a different role, and not every model is reachable through the OpenAI-compatible images API.

Two routes to a FLUX deployment

• Image API: the OpenAI-compatible /openai/v1/images/generations (and, for Kontext, /images/edits) endpoint. An app that already calls images.generate can switch to FLUX-1.1-pro or FLUX.1-Kontext-pro by changing only the model value. Check response handling: the FLUX samples read an image URL, while GPT-image deployments return b64_json.
• BFL provider-specific API: https://<resource>.api.cognitive.microsoft.com/providers/blackforestlabs/v1/<model-path>?api-version=preview. It supports every FLUX model and is the only route to provider parameters such as seed, guidance, steps, aspect_ratio and safety_tolerance. Availability varies by model: guidance and steps are documented for FLUX.2-flex and FLUX.2-pro, and aspect_ratio for FLUX.2-pro and FLUX.1-Kontext-pro, so check the model's parameter list.

The URL uses a model path, which is not the model ID: flux-kontext-pro (not FLUX.1-Kontext-pro), flux-pro-1.1, flux-2-pro, flux-2-flex. Using the ID in the path is a common mistake.

So any requirement that depends on a provider parameter, such as a fixed seed for reproducible output, means the BFL route; the Image API and the GPT-image models don't expose one. On FLUX.2-flex, guidance (1.5 to 10, default 4.5) sets prompt adherence, with low values weakening it, and steps trades detail for speed. Parameters don't change what a model accepts as input: a text-only model such as FLUX-1.1-pro can't edit an image whatever you set.

FLUX models don't get built-in content filtering at deployment time, so Microsoft advises configuring content safety at inference. That belongs to the responsible-AI topic of this domain.

Sora 2 generates video as an asynchronous job: videos.create starts a render and returns at once with a video object whose status is queued and which contains no video data. Renders take minutes, so the code follows a lifecycle:

1. Create with client.videos.create(model=..., prompt=..., size=..., seconds=...), using the OpenAI client against the resource's /openai/v1/ base URL.
2. Poll client.videos.retrieve(video_id) until the status is completed, failed or cancelled (it passes through queued and in_progress). The create_and_poll helper does both steps and blocks until the job ends.
3. Download the MP4 with client.videos.download_content(video_id, variant="video") and store it.

There is no URL in the create response and no streaming mode for video bytes. Remix is a separate operation for changing a finished video, not a way to start a queued job.

Jobs expire after 24 hours

Generated videos are available for up to 24 hours after the job is created (a completed job shows an expires_at time one day after created_at). After that, the only way to get the video again is a new job; nothing you do with the job object extends the window. The rule for any pipeline is to download each MP4 as soon as it completes and keep it in storage you control.

Designing a web app around long renders

Don't call create_and_poll inside an HTTP request handler: it blocks for the whole render and runs into gateway timeouts. The robust pattern is to call videos.create, return the video ID to the user, and let a background worker poll, download and store the MP4 so the user can come back to it. Keep the API calls on the server side, where the resource's credentials live.

A Sora 2 request is controlled by four main parameters on videos.create, which follows OpenAI's video schema:

Image-to-video with input_reference

input_reference turns a still into the opening frame of the clip, for example a landscape painting that starts to move. The image must have exactly the requested dimensions, so prepare it (resize or crop) to the size you request before upload. Because the API lists only the two sizes above, you adapt the image to the video size, not the other way round. (Microsoft's overview page also has a limitations list naming more sizes, apparently carried over from the earlier Sora API; the videos.create parameter table is the one to build against.) The reference image is also subject to Sora 2's content rules (see the last section), so it must not show a human face.

Don't mix in the older jobs API

Before Sora 2, Azure OpenAI video generation used a different REST API at /video/generations/jobs with fields such as width, height, n_seconds, inpaint_items and crop_bounds, and statuses such as succeeded. None of those are videos.create arguments. In the Sora 2 API you use size, seconds and input_reference, and the finished status is completed.

You edit a generated Sora 2 video with remix: client.videos.remix(video_id, prompt=...) takes the ID of a completed video and a new prompt. The service keeps the original's structure, motion, framing, scene transitions and visual layout, and applies the requested change. The result is a new video job with its own ID (its remixed_from_video_id points at the source), so you poll and download it like any other job.

One adjustment per remix

Microsoft's guidance is to limit each remix to one clearly articulated adjustment: narrow edits keep more fidelity to the source and are less likely to introduce visual defects. When several changes are needed, run them as successive remixes, each built on the last completed result. Packing all of them into one prompt invites drift from the original framing.

Editing exported frames one by one with the image edit API is not a video-editing workflow: each frame is edited independently, so the result loses temporal consistency, and the work is heavy.

Sora 2 requests most often fail because they break one of the API's current content restrictions, or because a reference image doesn't match the requested size. The Sora 2 API currently:

• allows only content suitable for audiences under 18;
• rejects copyrighted characters and copyrighted music;
• cannot generate real people, including public figures;
• rejects input images that contain human faces.

These rules apply to the prompt and to input_reference alike, and a remix is a generation too, so it can't be used to add a real person afterwards. A request is checked as a whole, so fixing the prompt doesn't rescue a non-compliant image, or the reverse. The workable design keeps the business goal and removes the restricted element from every input, for example by describing a generic, fictional character instead of a named celebrity.

When a job ends with status failed, check the job's error details: a content-policy rejection needs a changed prompt or image, not a retry. Request-level problems (an unsupported size, a reference image of the wrong dimensions) are rejected outright. Broader responsible-AI controls for visual content, such as content filtering, provenance and safety evaluation, are covered in this domain's responsible-AI topic.