AI-103 flashcards

Plan and manage an Azure AI solution

• Image in, text out needs a multimodal chat model; text in, image out needs an image generation model; vectors for search come only from an embedding model.
• A reasoning model fixes wrong multistep conclusions when the right facts are already in context; it does not fix missing knowledge and is slower per request.
• For narrow, high-volume tasks pick the cheapest model that meets the measured accuracy bar — a small or fine-tuned small model; Foundry Local runs SLMs on-device and offline.
• Model router chooses a model per request behind one deployment; its context limit is the smallest model's, so long inputs need a model subset of long-context models.
• Prefer a prebuilt Foundry Tool when it does the whole job: Content Understanding for schema fields with confidence, Speech custom models, Translator document translation, Language PII redaction.
• RAG for private, changing knowledge with citations; fine-tuning for style and format; structured outputs for a fixed JSON schema.
• Hybrid search (search_text plus vector_queries, merged with RRF) handles exact codes and meaning; the semantic ranker reorders top results using a semantic configuration and query_type semantic, with no re-embedding.
• Integrated vectorization needs an indexer skillset with Text Split and embedding skills plus a vectorizer on the vector profile; changing embedding models means re-embedding every document.
• Foundry IQ knowledge bases give agents shared, multi-source, permission-aware, cited agentic retrieval; file search covers uploaded files; the Azure AI Search tool uses an existing index as-is; Memory keeps per-user context.
• Function calling runs in your app, the OpenAPI tool lets the service call your API; MCP allowed_tools limits tools and require_approval always gates each call; prompt agent = configuration, hosted agent = your code, Responses API = agent defined in your app.

• A Foundry resource holds model deployments, networking and resource-level RBAC; Foundry projects hold agents and files. Separate resources isolate environments; projects under one resource share deployments.
• Upgrading an Azure OpenAI resource to a Foundry resource keeps its endpoint and keys; hub-based projects are Foundry (classic).
• The deployment type, not the resource region, decides where prompts are processed: Global (any region), Data Zone (a multi-country zone), Standard and Regional Provisioned (one Azure geography).
• Provisioned (PTU) gives predictable latency for steady volume; Batch is about half price with a 24-hour target and its own quota; Developer is a 24-hour, no-SLA deployment for testing fine-tuned models.
• service_tier: priority buys latency per token, default overrides a priority deployment, flex is half price but preemptible with no automatic fallback.
• Serverless API has built-in filters and data-zone options; managed compute hosts open-weight models, is global only and has no built-in filtering in preview.
• versionUpgradeOption: OnceNewDefaultVersionAvailable, OnceCurrentVersionExpired or NoAutoUpgrade (plus a migration plan before retirement).
• Deployment type is the deployment's sku.name, so Azure Policy on sku.name governs which types can exist.
• Standard agent setup: Cosmos DB for threads, Storage for files, AI Search for vector stores; capability settings cannot change on an existing project.
• Hosted agent versions are immutable, one version serves all traffic, scaling is per session, and each agent has its own Entra agent identity; pipelines use OIDC and an evaluation gate before prod.

• Quota is a TPM pool per subscription, region, model and deployment type; free capacity by lowering another deployment of the same model, because new resources don't add quota.
• The rate limiter counts prompt tokens plus max_tokens and enforces RPM over 1–10 second windows; fix 429s with right-sized max_tokens, paced sends and backoff with jitter.
• Provisioned gives predictable throughput and cost, Batch halves the price for work that can wait a day on separate quota, and spillover sends PTU overflow to a same-resource, same-version standard deployment.
• Budgets alert on actual or forecasted spend but never cap it; per-project chargeback uses Foundry's automatic project tag in Cost Management.
• Application Insights powers the Agent Monitoring Dashboard and tracing; continuous evaluation scores live traffic (project identity needs Foundry User), scheduled evaluation scores a fixed dataset to catch drift.
• A filtered prompt returns HTTP 400 content_filter, a filtered completion returns finish_reason content_filter, and an error inside content_filter_results means the filter didn't run.
• Without ground truth use Groundedness (answer vs context) and Retrieval (context vs query); with labels add Response Completeness and Document Retrieval.
• Per-document indexing failures live in the indexer's execution history; maxFailedItems -1 keeps a run going past bad files; replicas fix query throttling, partitions fix storage.
• Filter weak chunks on @search.rerankerScore (0–4), not the RRF @search.score, and use debug to see hybrid subscores.
• Keyless = managed identity + Foundry role + disableLocalAuth, with tokens for https://ai.azure.com/.default; user-assigned identities need AZURE_CLIENT_ID set to their client ID.

• A guardrail is a set of controls, each a risk plus intervention points plus an action; Microsoft.DefaultV2 can't be edited, and core harm controls can be overridden but not deleted.
• Pick the intervention point where the risky content exists: tool call for data an agent sends out, tool response for content a tool returns, output for the reply. Tool controls work only on moderation-supported tools such as OpenAPI, not MCP.
• Blocking level Low blocks only high-severity content and High blocks low and up; annotate-only or Off on outputs needs approval for modified guardrails.
• Document attack detection is off by default and needs documents wrapped in the documents delimiter; Spotlighting is an optional, token-costly, Chat Completions-only add-on.
• An agent's own guardrail fully replaces its deployment's; raiPolicyName applies to every caller of a deployment, x-policy-id overrides for one request.
• Hosted agents need rai_config with a full ARM ID (empty rai_config means DefaultV2, none means no screening); the invocations protocol also needs invocations_moderation with a response_mode covering every reply shape.
• Harm evaluators score 0–7 with a default threshold of 3; ungrounded attributes needs context, and the agentic evaluators and red-team categories need agent runs in the cloud.
• Trace everything in production but turn content recording off; images from Azure OpenAI carry signed C2PA Content Credentials automatically.
• Restrict tools with allowed_tools, gate the rest with require_approval (a never list keeps future tools gated), and use Task Adherence to escalate only misaligned calls.

Implement generative AI and agentic solutions

• AIProjectClient takes the project endpoint https://<resource>.services.ai.azure.com/api/projects/<project> and an Entra ID token credential only; get_openai_client() gives an OpenAI client with the same sign-in. Read project connections with connections.get_default or connections.get (include_credentials=True only when you need a stored key).
• On the v1 API, use OpenAI() with a base_url ending in /openai/v1/, no api-version, and a bearer token provider for keyless calls; 404 points at the path, 401 at the credential.
• The model parameter is always the deployment name; discover names with project.deployments.list instead of hard-coding them per environment.
• Check the model card: codex models are Responses API only, partner models such as DeepSeek work through v1 chat completions, and some models have no tool calling.
• Images go in input_image (URL, base64 data URI or file_id) and PDFs in input_file (file_url or file_id); detail low saves tokens on coarse checks.
• Chain turns with previous_response_id, or with store=False resend the output items plus reasoning.encrypted_content; stream=True fixes perceived latency, background=True with store=True handles long tasks.
• RAG is retrieve, augment, generate: label chunks with IDs, let the app map IDs to URLs, allow a not-found answer, and rewrite follow-ups into standalone queries before searching.
• Query vectors must come from the same embedding model as the index, and a vector query always returns k results; On Your Data has been retired, so use Foundry IQ or app-side retrieval.
• Your code runs every function call and returns one function_call_output per call_id; tool_choice controls whether and which tool is called, parallel_tool_calls=False handles dependent calls.
• Groundedness catches added claims, Response Completeness catches missing points, safety evaluators need no judge deployment, and dataset runs map {{item.field}} while target runs use {{sample.output_text}}.

• Instructions carry the agent's role, scope, tool decision rules and what to do on empty results; tool_choice=required (set by the app) is the deterministic way to force a tool call.
• Strict function schemas list every property in required and set additionalProperties to false; optional fields allow null. enum restricts values, required makes a field mandatory.
• The app runs functions: return a function_call_output per call with the matching call_id, all outputs in one request. Runs expire after 10 minutes, so slow work returns a job status.
• Conversations store every item but truncate the model input past the context window; store=false with app-held history is the only option when nothing may be persisted.
• Memory scope {{$userId}} uses x-memory-user-id if sent, else the caller's Entra identity — a backend serving many users must send the header. TTL only applies to stores created after TTL support.
• Azure AI Search tool: filter applies to every query; keyless access needs Search Index Data Contributor + Search Service Contributor on the Foundry account's managed identity. File search allows one vector store per agent and per conversation.
• OpenAPI tools need OpenAPI 3.0/3.1, letter/-/_ operationIds and JSON bodies; API keys live in a project connection plus securitySchemes. Toolboxes share tools through one MCP endpoint but can't hold function tools.
• The A2A tool keeps the calling agent in charge (incoming A2A on the target, Foundry Agent Consumer for the caller); handoff transfers the user. Connected Agents isn't in the new Agent Service, and portal workflows retire on 1 December 2026.
• Enforce approvals in the platform: require_approval {"never": [...]} gates everything unlisted; approval_mode="always_require" gates a function; request_info asks for input; checkpoints make waits survive restarts.
• Match evaluators to the failing step (Task Adherence, Tool Input Accuracy, Tool Output Utilization, Task Navigation Efficiency with matching_mode) and keep content recording to development.

• Lower temperature for repeatable output, raise it for variety; top_p is the alternative control — change one of the two, not both.
• Reasoning models reject temperature, top_p, penalties and max_tokens; use max_completion_tokens on Chat Completions and max_output_tokens on the Responses API.
• Reasoning tokens are hidden, billed as output and count toward the output cap; a small cap can return an incomplete or empty response, so reserve about 25,000 tokens while learning a workload.
• Reasoning effort controls thinking depth and verbosity controls answer length; none (gpt-5.1+) is for latency-critical work, minimal is original GPT-5 only, and gpt-5.1 defaults to none.
• Chain-of-thought prompting is for non-reasoning models; for reasoning models request a reasoning summary — other ways of extracting reasoning are unsupported.
• Self-critique needs a grounded critic, explicit criteria, a round limit and, when failures are rare, revision only for outputs that fail a fast check.
• Built-in evaluators return score, reason, label and a threshold (default 3); code-based custom evaluators are deterministic but have no network, endpoint-based ones can call your services.
• Set AZURE_EXPERIMENTAL_ENABLE_GENAI_TRACING before instrument(), get OpenAI clients after it, and remember @trace_function always records parameters.
• Prompt caching is automatic from 1,024 tokens with an identical prefix: static content first, dynamic content last; cached_tokens and reasoning_tokens are subsets of their totals.
• Use model router for managed per-prompt selection (Balanced, Quality, Cost, subsets); use your own classifier, cascade or rules engine when routing must be auditable, reach custom deployments or be deterministic.

Implement computer vision solutions

• dall-e-3 was retired on 4 March 2026; move to a GPT-image deployment, which uses the same images API but returns b64_json instead of a URL.
• GPT-image-2 is generally available and takes custom sizes (edges in multiples of 16); GPT-image-1, 1.5 and 1-mini need limited-access approval and offer three fixed sizes.
• Lower quality generates faster; streaming with partial_images shows previews sooner but doesn't finish sooner; output_compression only shrinks JPEG files.
• A transparent background needs background="transparent" with output_format="png".
• Edits are multipart/form-data with up to 16 PNG/JPG images under 50 MB; a mask must be a PNG of the same size whose editable area is fully transparent.
• input_fidelity=high preserves faces from input images, but gpt-image-1-mini doesn't support it.
• Only FLUX.1-Kontext-pro and FLUX-1.1-pro use the Image API; provider parameters such as seed, guidance, steps and aspect_ratio need the BFL API (availability varies by model), whose URL uses the model path, not the model ID.
• Sora 2 videos.create returns a queued job: poll, then download_content, and store the MP4 within 24 hours.
• The Sora 2 API lists size 720x1280 or 1280x720 and seconds 4, 8 or 12; input_reference must match size exactly.
• videos.remix edits a completed video while keeping its structure; make one clear adjustment per remix, and remember Sora 2 rejects real people and input images with faces.

• Images reach a vision model as input_image parts with a fetchable URL or a base64 data URL; a local file path or a file name in text gives the model no pixels.
• detail low means one 512 x 512 view; high adds detailed tiles for small text and fine features. It changes input resolution, never answer length.
• Caption length and focus come from instructions; one request can carry several images (up to 10 per chat request) for one combined caption.
• Grounded visual QA needs an answer-only-from-the-image instruction with an abstain path, high detail for fine evidence, and for video, retrieval over timed prebuilt-videoSearch segments with cited start times.
• Alt text: empty alt for decorative images, purpose-in-context plus informative text for others, and a short alt plus a long description for charts.
• Custom analyzers derive from prebuilt-document, -image, -audio or -video; image analyzers support generate and classify, and text-heavy images belong in document analyzers with source and confidence.
• Content Understanding runs on your Foundry deployments: resource defaults for everyone, a per-request modelDeployments override for one caller; the models property takes model names, not deployment names.
• Video: enableSegment false gives one value per video; true with one contentCategories entry (video allows only one) gives timed segments, optionally routed by analyzerId.
• returnDetails adds transcriptPhrases and cameraShotTimesMs; the analyzer samples about 1 fps at 512 x 512; analyzeBinary caps at 200 MB / 30 minutes, analyze by URL at 4 GB / 2 hours.
• Pro mode lived only in the retired 2025-05-01-preview API; agentic mode (config.workflow "agentic", 2026-06-01-preview) replaces it for document analyzers only.

• Analyze Image scores Hate, Sexual, Violence and Self-harm; the app compares each severity with its own threshold — the API never blocks.
• Image severities use only 0, 2, 4 and 6 (safe, low, medium, high); write image thresholds against those values.
• Images must be at most 4 MB and 50 x 50 to 7,200 x 7,200 pixels; animated images are judged by their first frame only.
• With a blobUrl, the Content Safety resource's system-assigned identity reads the blob and needs Storage Blob Data Contributor or Owner.
• The image-with-text API with enableOcr judges an image and its text together, but only up to 1,000 characters; score longer text with Analyze Text.
• Guardrail blocking levels: Low blocks high only, Medium blocks medium and high, High blocks everything above safe; safe content is never blocked.
• Turning a harm control off on outputs needs modified-guardrail approval; on inputs it doesn't.
• Prompt Shields reads text, not pixels: extract image text and send it in the documents array, then act on documentsAnalysis[].attackDetected.
• Content Credentials are an automatic, signed C2PA manifest seen only with verification tools; a visible label must be composited by the app.
• New or organisation-specific symbols need a deployed custom categories (rapid) image incident; blocklists and standard custom categories are text-only.

Implement text analysis solutions

• Standard entities: prebuilt NER. Own categories with labels, offsets and retraining: custom NER. Own categories, no labels, values only: a prompted model with a strict schema.
• Strict structured outputs need every property in required and additionalProperties false on every object; express optional values as a null union.
• Fixed label lists belong in an enum; instructions and temperature 0 reduce but never prevent off-list values.
• Content larger than the context window is chunked and summarized in levels; truncation, partial input and max_output_tokens all lose coverage.
• Sentiment analysis and opinion mining are legacy Azure Language features for existing apps; tone, sarcasm and new builds use a language model with a schema.
• Content Safety text analysis screens any text: EightSeverityLevels gives 0–7, blocklists catch policy terms, haltOnBlocklistHit (default false) skips further analysis on a hit. Model guardrails see only model traffic.
• PII detection returns redacted text plus each entity's category and confidence score; categories_filter narrows what is masked.
• In Translator 2026-06-06, deploymentName defaults to general NMT; tone and gender need an LLM deployment; allowFallback false makes unsupported pairs fail; LLM translation is unavailable behind a private endpoint.
• referenceTextPairs (up to five) override adaptiveDatasetId; a Custom Translator model is the NMT-only route to your terminology.
• Try instructions and examples first, enforce with a schema, and fine-tune only when long per-call examples are the cost problem.

• Live audio → real-time Speech SDK; one file, synchronous → fast transcription (display form only); many stored files → batch transcription (lexical form available, custom model without an endpoint).
• Select a voice with speech_synthesis_voice_name; SSML (prosody rate, break time, styles) needs speak_ssml_async; Raw formats have no header, and 8 kHz mu-law is North American/Japanese telephony while A-law is European.
• Output pronunciation is fixed on the TTS side: sub for a one-off spoken alias, a case-sensitive PLS custom lexicon (phoneme or alias) for every reply, custom_lexicon_url in Voice Live.
• Voice Live is the managed speech-to-speech layer: noise suppression, echo cancellation, end-of-turn detection, avatars; agent mode takes agent_name + project_name and requires Entra ID.
• Phrase lists are runtime, no-training boosts for modest lists; they don't work in batch transcription or with whisper/gpt-4o-transcribe in Voice Live — use custom speech or azure-speech instead.
• Custom speech data follows the error: text for vocabulary, audio + human-labeled transcripts for noise and accents (and for WER testing), plus pronunciation and display formatting rules.
• A custom speech model needs a custom endpoint and endpoint_id for real-time recognition and speech translation, batch references it directly, and fast transcription/LLM Speech take none; per-language custom models go in SourceLanguageConfig; Voice Live custom_speech maps locale to model ID, azure-speech only, same Foundry resource.
• Professional custom voice = one brand voice from studio data (azure-custom + endpoint_id); personal voice = a user's own voice from a short sample with consent (azure-personal).
• Audio reasoning needs the audio itself (input_audio parts to gpt-4o-audio-preview); Realtime API uses WebRTC for clients with a backend-minted ephemeral token, WebSockets server to server.
• Speech SDK translation: locale source, language-code targets, event synthesis for one target only; Live Interpreter adds open-range detection and personal voice; whisper translates only into English.

Implement information extraction solutions

• Supported source and no upload code means an indexer on a schedule; blob change detection uses LastModified automatically.
• Native blob soft delete detection needs versioning off and must exist from the first run; deletions take effect on the next indexer run.
• fieldMappings read source fields before skills run; outputFieldMappings send skill outputs to the index. Use base64Encode to turn a path into a valid key.
• Scans need imageAction plus the OCR skill, and the Text Merge skill is what puts image text into the content that gets chunked and embedded.
• For new layout-aware skillsets use the Azure Content Understanding skill: Markdown tables, cross-page chunks, locationMetadata page ranges, and allowSkillsetToReadFileData for file_data.
• Index projections: never map the parent key, use a keyword-analyzed string key and a filterable string parent field, and skipIndexingParentDocuments for chunk-only indexes.
• Embedding skill dimensions must equal the vector field's; chunk anything over about 8,000 tokens; empty apiKey and authIdentity means the system-assigned identity.
• Built-in skills are free for 20 documents per indexer per day; a keyless AIServicesByIdentity attachment removes the limit with no region requirement.
• Custom Web API responses must echo every recordId; tune batchSize, timeout (max PT230S) and degreeOfParallelism (max 10) together.
• Verbalize diagrams that must be explained, embed images directly for look-alike search; use Content Understanding audioSearch and videoSearch analyzers for media.
• One Azure AI Search tool targets one index; a knowledge base over several indexes connects through a RemoteTool MCP connection, with a per-request user token for permission trimming.

• Content extraction analyzers need no generative model: prebuilt-read for text and barcodes, prebuilt-layout for structure, prebuilt-digitalParse for born-digital files and, in the GA API, embedded metadata.
• prebuilt-documentSearch is the RAG ingestion analyzer: markdown plus figure descriptions, Chart.js and Mermaid analysis, annotations and a summary, with no schema to write.
• Custom analyzers derive only from prebuilt-document, prebuilt-image, prebuilt-audio or prebuilt-video; turn on the figure options yourself to get documentSearch-style figures.
• enableOcr, enableLayout, enableFormula and enableBarcode default to true; enableFigureDescription and enableFigureAnalysis default to false; tableFormat defaults to html.
• Markdown encodes checkboxes as ☒/☐, page numbers, headers, footers and breaks as HTML comments, sections as headings and formulas as LaTeX.
• extract returns printed values, classify returns one enum value, generate infers; write detailed, affirmative, same-language descriptions with aliases, and use arrays of objects for repeated rows.
• Confidence and grounding come from estimateFieldSourceAndConfidence (analyzer) or estimateSourceAndConfidence (field); source locates the value on the page, spans in the markdown.
• Labeled samples come after refined descriptions, one per underperforming layout, measured on a fixed test set; they can't fix generate fields or OCR misreads.
• contentCategories with enableSegment splits and routes mixed files; add an 'other' category, use segmentPerPage for one form per page and omitContent for routed results only.
• Analyze is asynchronous (202 plus Operation-Location to poll); read contents[0].markdown and fields, and copy prebuilt analyzers into your own for production.

Yes — the whole AI-103 deck is free and needs no account, as is every AI-103 revision lesson. Create a free account to save progress and drill practice questions; full-length mock exams are a one-time unlock per certification.