AIP-C01 GenAI Solution Design: Architecture, Proofs of Concept and Standards

Pick the cheapest technique that fixes the actual gap: prompt engineering for output format and behaviour the model can already produce, retrieval-augmented generation (RAG) for facts that change or must be cited, fine-tuning for a style or behaviour that prompting cannot hold and for which you have labelled examples, and continued pre-training for deep domain vocabulary learned from large amounts of unlabelled text. Requirements decide this, not ambition: a design that trains a model to fix a layout problem, or bakes daily-changing prices into weights, fails a stated constraint.

Three reasoning rules follow from the table. First, fix format with the prompt: if the model understands the content but the layout varies, add the headings and a one-shot example to the template. Second, facts that change or must be cited belong in retrieval, because knowledge absorbed into weights is stale after the next edit and cannot point to the page it came from. Third, techniques combine: when one requirement is about current facts and another is about voice, the design retrieves the facts at request time and fine-tunes for the voice, because each technique fixes only its own gap. Relocating instructions from the prompt into a retrieval source does not change behaviour, because retrieved text reaches the model as prompt text too.

Host in Amazon Bedrock when a managed, serverless API is enough, and on an Amazon SageMaker AI endpoint when you need your own container, your own inference code or a chosen GPU instance type. Bedrock gives you three ways to run a model: catalog foundation models on demand, models customised inside Bedrock (fine-tuning or continued pre-training jobs that produce a Bedrock-managed custom model), and Custom Model Import for weights you trained elsewhere.

Custom Model Import bills per active model copy in five-minute windows, scales copies to zero when idle and leaves only a small monthly storage charge, so it suits bursty or idle-prone traffic. Its limits are what push a design to SageMaker AI: you hand over weights, not container code, so a proprietary library that must run beside the model has no place to live; an architecture outside the supported list (for example custom attention layers) cannot be imported; imported models are invoked on demand rather than through batch inference; and an import job cannot be declared in AWS CloudFormation, so it is scripted with the API or CLI. Custom Model Import reuses weights you already have, whereas a Bedrock customization job trains again from a catalog base model. SageMaker JumpStart can deploy open-weight models to an endpoint quickly, but every endpoint is billed infrastructure you must delete when the test ends.

Choose the invocation pattern from three facts: whether someone is waiting for the answer, how large and slow each request is, and how traffic arrives. Synchronous calls serve users in the request; batch processing serves files of work nobody waits on; asynchronous endpoints sit between the two.

Two near-twin traps recur. Serverless inference scales to zero but has no GPU, so it cannot host a typical LLM; asynchronous inference scales to zero and supports GPUs but cannot return the answer inside the caller's request. And a real-time endpoint with target-tracking auto scaling keeps at least one instance running, so it pays through every idle night.

Amazon Bedrock lets you trade price against speed and assurance per request with the service tier (the service_tier setting), plus capacity commitments for workloads that must not throttle.

Read the requirement carefully: fastest without a reservation points to Priority; assured capacity for a known baseline over a few months points to Reserved; cheapest per token, per-request points to Flex, while cheapest for files of prompts points to batch inference. Only a reservation assures capacity: a quota is a ceiling, not a guarantee, and retry logic recovers from throttling rather than preventing it.

When a model call can take minutes or be throttled, take it out of the client's request: accept the work, put it in a durable buffer, process it in the background and tell the client when it is done. Longer timeouts only keep connections open longer; a throttled synchronous call still fails with nothing holding the work for a retry.

• Queue plus status store. Amazon API Gateway writes each job to Amazon SQS and returns a job ID; a Lambda consumer calls the model, retries through throttling and records status and result (for example in DynamoDB). Nothing is lost during throttling peaks, and the queue smooths bursts.
• Push the result instead of polling. An API Gateway WebSocket API keeps a connection open so the back end can post the finished result to that connection the moment it exists. Amazon SNS delivers to subscribed endpoints such as email, SMS, HTTP(S) endpoints, SQS queues and Lambda functions, so it has no way to address one client's open session.
• Long, multi-step jobs. An Amazon EventBridge rule on S3 object creation starts an AWS Step Functions Standard workflow, a Map state fans out over chunks (chapters, pages), each Bedrock task has its own retry policy, and a final step publishes to SNS. Standard workflows run for up to a year; Express workflows stop at five minutes and a Lambda function at 15 minutes, so neither can carry a job that takes most of an hour.

Provisioned or reserved capacity can remove throttling, but it does not make a multi-minute call suitable for a synchronous web request, and it adds standing cost.

Use a workflow when you can draw the path in advance and an agent when the path must be decided at run time. Regulated processes with fixed, audited steps belong in an AWS Step Functions Standard workflow: it runs steps in a defined order, records every state's input and output in its execution history, calls Bedrock through a service integration and pauses for a human with a task token (.waitForTaskToken). Express workflows suit short, high-volume runs and cannot wait days for an approver.

The strongest designs are often hybrids: a workflow fixes and records the audited path, and one task inside it calls an agent for the single open-ended step (for example, a research step in which each lookup depends on what the previous one found). Encoding every combination with Choice states is brittle; handing the whole process to one agent loses the enforced order auditors need. A knowledge base retrieves text but does not act; a chain of Lambda functions makes ordering, retries and audit records custom code. Building agents in detail is Task 2.1; here the decision is which pattern the requirements call for.

Start with an Amazon Bedrock knowledge base when the goal is the least operational overhead: it manages ingestion from sources such as S3, chunking, embedding, the vector index and retrieval, and its RetrieveAndGenerate API returns answers with citations. Build your own chunking, embedding and vector store pipeline (for example Lambda plus Amazon OpenSearch Service, or a framework on Amazon ECS) only when a requirement the managed service cannot meet justifies owning that code and infrastructure.

Not every question is a similarity search. When answers must be computed from warehouse tables (totals, rankings, period-over-period changes), a knowledge base connected to a structured data store such as Amazon Redshift turns the natural-language question into SQL (the GenerateQuery capability), runs it against the current tables and generates the answer from the results. Embedding table rows as text chunks retrieves similar passages but cannot aggregate, and prompting a model with the schema and running its SQL yourself is the custom code the managed feature removes. Vector store and retrieval design in depth belong to Tasks 1.4 and 1.5.

A proof of concept (PoC) answers one question cheaply before money goes into a production build: can this work, how well does it perform, and is it worth it? The fastest feasibility checks create nothing that must be cleaned up.

• Bedrock console playgrounds let anyone, including non-coders, try prompts and inference parameters such as temperature against catalog models with nothing deployed.
• Querying one document without a knowledge base: RetrieveAndGenerate with an external source accepts a single document from S3 or as bytes (up to 10 MB) and returns cited answers, with no vector store or sync job.
• Avoid PoC paths that leave billed resources behind (a quick-created vector store, a JumpStart endpoint, a notebook) or take far longer than the question deserves (fine-tuning).

To choose between models, test them on your own data at scale, not on public benchmarks or a handful of playground prompts. Amazon Bedrock model evaluation offers three job types:

An LLM-as-a-judge job evaluates one generator model, so comparing two candidates means one job per model on the same dataset. Live A/B tests on customers are production experiments, not PoC evidence. Building full evaluation systems is Task 5.1.

A PoC must report measured numbers, and each number has a specific source. Use the wrong one and the report overstates latency, cost or capacity.

Throughput validation. Bedrock deducts input tokens plus max_tokens from the tokens-per-minute quota when a request starts and returns the unused part when it ends (some models also apply an output-token burndown multiplier); billing follows the tokens actually used. A load test that throttles while measured usage sits well below quota usually means max_tokens is set far above real output. Right-size it, re-run the test at projected peak and compare InvocationThrottles before asking for a quota increase sized to real demand. Streaming does not change the reservation, and more Lambda concurrency only reaches the quota sooner.

Cost attribution. Create an application inference profile per team or PoC, tag it, invoke through its ARN and activate the tags as cost allocation tags; Cost Explorer then splits model spend. Base foundation models are AWS-owned and cannot be tagged. Bedrock can also allocate cost by tags on IAM principals (users and roles), but that cannot separate teams that share one execution role.

Business value. Feasibility and latency are not value. Prove value with a business KPI against a baseline, ideally a pilot group compared with a control group on the same work, measured in the unit the business already tracks (time or cost per case, cases completed per day). Opinions and usage counts show adoption, not outcome. Project cost as billed tokens per transaction from a production-like run, times volume, times price; a PoC's total bill mixes in exploratory traffic that production will not have, so it is a poor basis for projection.

Promoting to production. Rebuild console-made PoC resources (knowledge base, guardrail, prompts) as infrastructure as code (AWS CloudFormation or the AWS CDK) and deploy them through the pipeline; there is no console export that moves them between accounts. Resources CloudFormation does not support, such as a Custom Model Import job, are scripted with the API or CLI in the same pipeline. Model availability and Bedrock quotas are set per account and per Region, so a quota raised in the sandbox says nothing about the production account in another Region: check availability and request increases there.

Standardised components make every team build and review GenAI workloads the same way: shared review content in the AWS Well-Architected Tool, and reusable, versioned infrastructure for building.

The Generative AI Lens

The Generative AI Lens extends the Well-Architected Framework for foundation-model applications across lifecycle stages from scoping and model selection through customization, development, deployment and continuous improvement. Its six pillars include, for example: Security (protecting endpoints, mitigating harmful outputs and excessive agency, auditing, securing prompts); Reliability (throughput, failure handling, versioned artifacts, distributed inference); Operational excellence (output quality, operational health, traceability, lifecycle automation); Performance efficiency (capturing and maintaining model performance); Cost optimization (cost-optimized models, prompts engineered for cost, efficient vector stores and agent workflows); and Sustainability. Traditional ML that trains its own models with no foundation model is reviewed with the Machine Learning Lens instead.

AWS distributes the Generative AI Lens as a JSON file in its public custom-lens GitHub repository, so it does not appear in the tool's built-in lens catalog: you import it as a custom lens, publish it, then share it. Custom lenses are shared through the Well-Architected Tool's own sharing (not AWS RAM) with accounts, IAM users, an organization or OUs; sharing with the organization requires enabling AWS Organizations access in the tool first. When you change a custom lens, a major version notifies workloads and waits for owners to upgrade, while a minor version applies automatically without notification.

Reusable building blocks

The deciding phrase is often accounts created later: only service-managed StackSets with automatic deployment cover them without further action. Copying a reference stack into each repository starts correct but drifts, with no upgrade path. Pair building and reviewing: reusable versioned code for what teams deploy, and a shared custom lens or review template for how they are assessed.