SAP-C03 AI and ML Integration: Multi-Account Bedrock, AgentCore Agents, SageMaker AI Inference and RAG Design

Four managed building blocks cover almost every AI integration a Professional-level architect designs, and the first decision is which one owns the job: Amazon Bedrock when the application calls a managed foundation model, Amazon Bedrock AgentCore when an agent needs hosting, tools, memory and secure access to systems, Amazon SageMaker AI endpoints when you serve your own trained or open-weight model, and Amazon Bedrock Knowledge Bases when answers must be grounded in your documents or data (retrieval-augmented generation, RAG).

The exam keeps this task at the architecture level. Model training, fine-tuning and data-science workflows are out of scope. Related decisions live in other tasks: Amazon Bedrock Guardrails and AgentCore Identity (Task 2.5), model invocation logging (Task 2.3), AI cost levers such as batch inference and prompt caching (Task 3.1), token chargeback with application inference profiles (Task 3.3), Bedrock evaluations and AI monitoring (Domain 5). This lesson names them only where they are the wrong answer to an architecture question.

There are two sound multi-account patterns for Amazon Bedrock, and the requirements decide between them: a central AI gateway when one team must be the single enforcement point for quotas, model choice and usage reporting, or direct access from each workload account when units must be isolated from each other's spikes, pay for their own usage, and nobody wants to run a shared service.

Making the gateway the only path. A gateway enforces nothing if a workload-account administrator can call Bedrock directly. Pair it with an SCP on the workload OUs that denies the Bedrock inference actions, so the controls hold even for account administrators. Detective measures (central model invocation logging, AWS Budgets alerts) report usage after it happens; they do not enforce a quota or an allow-list. Giving each workload account its own IAM role with Bedrock permissions, or its own inference profile, simply recreates direct access.

Reaching the gateway privately. When workload VPCs have no internet access, overlapping CIDR ranges, or a ban on VPC peering, use AWS PrivateLink rather than routing: make the gateway a private REST API, create an execute-api interface VPC endpoint in each consuming VPC, and allow those endpoint IDs in the API's resource policy (condition on aws:SourceVpce). PrivateLink exposes a service, not a network, so it needs no routes between VPCs and overlapping CIDRs do not matter. Routing-based connectivity requires non-overlapping address space, and any endpoint that points a workload directly at Bedrock is another path around the gateway, not a way to reach it.

Use a service control policy that denies the Bedrock inference actions (bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream, which also govern the Converse and ConverseStream APIs) on every model resource except the approved model ARNs. Attached to the root or to the relevant OUs, an SCP applies to every current and future member account from creation and constrains account administrators too, with no per-account work.

Why other controls fall short:

• Denying aws-marketplace:Subscribe is not enough. Amazon Bedrock subscribes to a Marketplace-listed model automatically on its first invocation, so a Subscribe deny does not stop that first call, and many models (for example Amazon's own) have no Marketplace product ID to deny at all. Block inference, not subscription.
• Permissions boundaries attach per principal in each account. They need work in every new account, and an administrator can create a role without one.
• CloudTrail plus EventBridge revocation is detective: the unapproved inference has already run.

SCPs and cross-Region inference. When an application invokes an inference profile, Bedrock authorizes the request in the destination Region it routes to, and the identity policy must allow both the profile and the underlying model there. A common Region-restricting SCP (deny when aws:RequestedRegion is not the home Region) therefore starts producing AccessDeniedException on many requests. The fix depends on the profile type: for a geographic profile, the SCP must allow the Bedrock invoke actions in every destination Region listed in the profile; for a global profile, AWS documents allowing aws:RequestedRegion with the value unspecified. The general rule: an explicit deny in an SCP overrides any allow lower in the hierarchy, so the fix always belongs in the SCP itself. Cross-Region inference can use its destination Regions without you enabling them first.

Workloads in private subnets reach Amazon Bedrock through interface VPC endpoints (AWS PrivateLink), one per API they call, with private DNS enabled so the SDK resolves the normal endpoint name to private addresses. No NAT gateway or internet gateway is needed. Bedrock has no gateway endpoint; gateway endpoints exist only for Amazon S3 and DynamoDB.

An application that calls a model and queries a knowledge base therefore needs both bedrock-runtime and bedrock-agent-runtime; the control-plane endpoints are for configuration tools, not runtime traffic.

Raising on-demand throughput. When on-demand requests are throttled at peaks, cross-Region inference spreads them across several Regions with nothing more than a change of model identifier to an inference profile ID, and no capacity commitment.

• A geographic profile (for example US, EU or APAC) keeps processing inside that geography. It is the choice when data residency rules apply.
• A global profile can route to any supported commercial Region worldwide. It offers the most headroom, and AWS prices it below geographic routing, so it suits workloads with no residency obligation.

The alternatives fail on stated constraints: Provisioned Throughput buys dedicated capacity but requires a term commitment and is purchased in one account; batch inference is cheaper per token but asynchronous, so it cannot answer users in real time; deploying a second copy of the application in another Region is a far larger change than switching to a profile.

Amazon Bedrock AgentCore Runtime is the managed host for agents and MCP servers: you deploy existing agent code built with any framework (LangGraph, Strands Agents, CrewAI and others) and any model, without managing servers or container clusters. Two properties decide most questions: sessions can run for up to 8 hours, and each session runs in its own microVM, which is terminated and its memory sanitized when the session ends, so one client's data is isolated from another's at the compute level.

Because sessions are deliberately ephemeral, a longer session is never a substitute for durable memory (see AgentCore Memory below).

Networking. A runtime in the default public network mode has no presence in your VPC. To call private resources, such as an API behind an internal Application Load Balancer, configure VPC network mode with private subnets and a security group: AgentCore creates elastic network interfaces there, and the target's security group must allow traffic from the runtime's security group. Keep the direction straight:

• VPC network mode = outbound from the agent into your VPC.
• The AgentCore interface VPC endpoint = inbound, letting clients in your VPC invoke the runtime privately. It gives the agent no path to your resources.
• A NAT gateway only gives private subnets outbound internet access, and making the ALB internet-facing breaks the requirement that it stays private.

An agent is only as useful as its tools, and AgentCore supplies managed versions of each kind: Gateway for existing APIs and functions, Code Interpreter for running code, Browser for websites without APIs, Memory for context across turns and sessions, and a Bedrock knowledge base as the retrieval tool for documents.

AgentCore Gateway

Gateway turns existing capabilities into Model Context Protocol (MCP) tools behind one endpoint, with no MCP server code to write or host. Targets include OpenAPI and Smithy specifications, AWS Lambda functions and existing MCP servers. It handles inbound authentication (for example OAuth tokens from your identity provider) and outbound credentials to the targets, so agents built with different frameworks share the same tools. For large catalogs (hundreds of tools), create the gateway with semantic search enabled: agents then call a built-in search tool (x_amz_bedrock_agentcore_search) that returns only the tools relevant to the task, instead of listing every definition and exhausting their context. Semantic search is chosen when the gateway is created.

Near-twins that fail: hosting a hand-written MCP server on Runtime works but is the code you wanted to avoid; Amazon API Gateway gives a REST front door but does not speak MCP; Bedrock Agents action groups serve only that one Bedrock agent; a knowledge base of tool descriptions is a separate catalog that drifts from the real targets.

Built-in tools and memory

A typical agent composes them: actions (check an order) through a Gateway target, knowledge (policy documents with source attributions) through a knowledge base. Agent authentication and credential handling (AgentCore Identity) and content safety (Guardrails) are covered in Task 2.5.

Choose the SageMaker AI inference option from four facts in the scenario: payload size, processing time, traffic pattern (steady, bursty, idle) and whether a caller waits for the answer.

Read the limits as hard boundaries: a request that exceeds an option's payload or time limit, or needs hardware it does not offer, rules that option out however attractive its pricing. Equally, a design where someone waits for the answer needs a synchronous option, and work with no waiting caller is cheaper offline or queued. Prefer the option that already provides a capability (queueing, scale to zero, job-based compute) over building it around a different one.

Hosting many models on shared infrastructure

One endpoint per model is the expensive baseline these options replace; separate serverless endpoints per tenant remove idle cost but still leave thousands of endpoints to operate.

Update a SageMaker AI real-time endpoint with a blue/green deployment guardrail: SageMaker AI provisions a new (green) fleet, shifts traffic to it, watches Amazon CloudWatch alarms during a baking period, and automatically rolls all traffic back to the old (blue) fleet if an alarm fires. There is no downtime and no engineer in the loop.

Testing before any customer sees the new model. A shadow test deploys the candidate as a shadow variant that receives a copy of live production requests on its own instances; its responses are captured for comparison (latency, errors, predictions) and never returned to callers. That differs from a second production variant with a traffic weight, which answers real customers, and from replaying captured requests through batch transform, which checks predictions but not behaviour under live load on the endpoint's instance type.

A common two-phase design is a shadow test first, then a blue/green release with linear or canary shifting and alarm-based auto-rollback. Weighted production variants that an engineer adjusts by hand, or separate endpoints behind Route 53 weighted records, lack automatic rollback (and SageMaker AI endpoints are invoked through the SageMaker runtime API, not by DNS name).

RAG ingestion is the pipeline that turns source content into searchable chunks: a data source is synced, each document is parsed into text, split into chunks, embedded, and written with its metadata to a vector store. Amazon Bedrock Knowledge Bases runs all of it as a managed service, so most answers keep you inside it rather than building a custom embedding pipeline that bypasses its chunking and metadata handling.

Data sources and keeping them current

• Incremental sync. Each StartIngestionJob on an existing S3 data source processes only objects added, modified or deleted since the previous sync. Running it on a schedule (EventBridge Scheduler and a small Lambda function) or on events keeps the index fresh at a cost proportional to change. Deleting and recreating the data source throws the sync state away and re-embeds everything.
• Cross-account S3. A knowledge base can ingest from a bucket in another account in the same Region without copying: the data source names the bucket and its owner account ID, the knowledge base service role has identity permissions, and the owning account grants it in the bucket policy (s3:ListBucket, s3:GetObject) and, for SSE-KMS with a customer managed key, in the key policy (kms:Decrypt). The knowledge base does not assume a role in the other account.
• Data deletion policy. With the default (delete) policy, deleting a data source removes its vectors from the store.

Parsing

The default parser extracts text. When answers live in tables, diagrams or scanned pages, choose advanced parsing with a foundation model (or Amazon Bedrock Data Automation) so that content is extracted at all. Chunking and retrieval settings cannot surface content that parsing never extracted.

Chunking

Small chunks match precisely but carry little context; large chunks carry context but match poorly. Hierarchical chunking is the strategy that separates the two. The chunking strategy cannot be changed after a data source is created. To switch without downtime, create a second data source on the same content with the new strategy, sync it while the old chunks keep answering, then delete the original data source so its chunks are removed. Chunking is never a query-time setting.

Choosing the vector store

Capacity-based stores (OpenSearch Serverless capacity units, Aurora capacity ranges, Neptune Analytics memory-optimized units) cost more than object storage for a rarely queried, billion-scale index. When each tenant's KMS key must protect its embeddings with independent revocation, a shared vector store under one key cannot meet it: the silo has to extend to a knowledge base and vector store per tenant, both under that tenant's key.

Retrieval quality and safety are mostly query-time configuration of the knowledge base APIs, and each symptom maps to one feature.

Match the fix to the stage that is failing: if the right content is retrieved but ranked badly, change ranking; if only part of the question is searched, change how the query is formed; if the content was never extracted, change parsing. Settings that act on a different stage, such as returning more results or changing generation parameters, leave the underlying retrieval problem in place.

Retrieve or RetrieveAndGenerate. RetrieveAndGenerate retrieves and then generates with an Amazon Bedrock model or inference profile, returning citations. Retrieve stops after retrieval and returns chunks, scores, source locations and metadata, so the application can prompt any model it chooses, including one hosted on a SageMaker AI endpoint, while still using the managed query embedding and vector search.

Tenant isolation in a pooled knowledge base. To serve many tenants from one knowledge base, attach a tenant attribute to every document with a <file>.metadata.json file next to it in S3 and sync, then have the backend add an equals filter on that attribute, taken from the authenticated identity, to every Retrieve request. The filter runs inside the vector search, so nothing a user types can widen it. Implicit filtering, guardrail denied topics and prompt-template instructions all depend on the model or the prompt, so a crafted prompt can get around them; none is an isolation control. When tenants require their own encryption keys over embeddings, move from this pooled model to a knowledge base per tenant (previous section).

Structured data. Questions that need exact aggregates ("net sales by region last quarter") need SQL, not similarity search. A knowledge base with a structured data store converts the natural-language question into SQL, runs it on an Amazon Redshift query engine (provisioned or Serverless) and answers from the result. The data can sit in Redshift or in S3 tables registered in the AWS Glue Data Catalog, queried in place. Amazon Athena is not a supported query engine. When AWS Lake Formation governs the catalog tables, the knowledge base service role needs Lake Formation Describe and Select grants plus the IAM permission lakeformation:GetDataAccess; Glue Data Catalog stores authenticate with IAM, and Secrets Manager credentials apply only to Redshift databases. Vectorizing exported tables loses freshness and cannot compute totals, and an agent action that sends questions to the Redshift Data API still needs your own text-to-SQL code.