Choosing ML, Foundation Model and RAG Approaches on AWS (MLA-C02)

Choose the least custom approach that meets the requirement: a pre-trained AWS AI service when it already does the exact task, a foundation model (FM) when the output is open-ended language or multimodal reasoning, a traditional ML model when you have labeled tabular or high-volume data and need a cheap, fast, repeatable prediction, and a fully custom model only when nothing managed fits. Each step down that list adds control and effort.

The common trap is reaching for an FM because it is the modern option. A recurring score for every customer, learned from labeled historical columns, is a supervised tabular problem: a gradient-boosted tree such as XGBoost learns from the labels and scores large populations cheaply, whereas prompting an FM per row is slow, costly and ignores the labels. The same logic applies to high-volume text classification into a fixed set of categories when a large labeled set already exists: a small supervised text classifier (for example the SageMaker AI BlazingText algorithm in supervised mode, or a custom text classification model built without code in Amazon SageMaker Canvas) costs a fraction of per-token FM inference, even discounted batch inference, because the FM bill repeats for every item on every run. Clustering (k-means) is not a substitute for classification: it finds groups but cannot map them to your fixed business categories.

Conversely, when a pre-trained AI service already returns exactly what is asked (for example the four sentiment classes positive, negative, neutral and mixed from Amazon Comprehend), any option that trains or fine-tunes a model is more effort and needs labels you may not have.

Hosting is part of build versus buy

Host a custom model where its architecture, its runtime dependencies and its traffic pattern fit: Amazon Bedrock Custom Model Import for supported open architectures called on demand, a SageMaker AI real-time endpoint when you need a custom container or steady GPU capacity, and SageMaker AI Serverless Inference only for small models that can run on CPU.

Deployment mechanics (endpoint types, inference components, auto scaling) are Task 3.1; here the point is that hosting is part of the build-versus-buy decision.

Match the algorithm to the problem type and to whether you have labels, then let interpretability and latency requirements narrow the field. Accuracy is only one criterion: a model that is marginally more accurate but cannot satisfy a published-explanation rule or a latency budget is the wrong model.

Interpretability is a requirement, not a preference

Different explanation tools answer different questions, and regulators often require a specific one. A linear model's coefficients are global and fixed: each feature raises or lowers the score by the same amount for everyone. SHAP values (for example from Amazon SageMaker Clarify) are per-prediction attributions, so the contribution of a feature varies from applicant to applicant. A global feature-importance ranking from a tree model says which features matter most but gives neither the direction nor the size of their effect. Monotonic constraints in XGBoost guarantee the direction of a feature's effect but not a constant size. Match the tool to the wording of the rule: a requirement for one fixed, published weight per feature is met only by a linear model, and a small accuracy gap is the accepted price of meeting a mandatory rule.

Solution templates and latency

Amazon SageMaker JumpStart offers pre-trained models and solution templates that package a common use case end to end, which shortens time to a working baseline. For latency, smaller and simpler models (linear models, shallow trees) answer in milliseconds on CPU; large deep networks and FMs need GPUs and take longer. Domain-specific performance should be measured on your own data rather than assumed from general benchmarks.

Select an FM in two passes: first apply hard filters that rule models in or out, then compare the survivors on your own task for quality, latency and cost. Comparing price or benchmark scores before checking the hard filters wastes time on models that cannot do the job.

Hard filters (pass or fail)

• Input and output modality: if users send photos with text, the model must accept image input. Check the input side specifically: other capabilities a model has, on the output side or in how it delivers responses, do not substitute for accepting the input type you need.
• Context window: the maximum input tokens per request. If a long document must be read whole in a single request, the context window decides eligibility before anything else. The maximum output tokens is a separate limit on the length of the response.
• Languages the model supports well.
• Features the application needs, such as tool use, streaming, or support for Bedrock customization methods.
• Regional availability where data residency rules apply.

Comparing the survivors

Run a representative set of your real inputs through each candidate and compare output quality, latency and token cost on the same set. General-purpose benchmark scores and model size are proxies that may not reflect your task. Amazon Bedrock provides model evaluation jobs for doing this comparison systematically; how to run and score them is covered in Task 2.3.

Choose an embedding model by the modality of what you need to search, the languages involved, and the vector size you can afford; the same model must embed both the documents and the queries, because retrieval compares vectors in one shared space. Only embedding models produce vectors; a text generation model returns text, not an embedding.

The table covers common choices, not the whole catalog: newer models such as Cohere Embed v4 are also available in Amazon Bedrock, so check the current model list and each model's modalities, languages and input limits.

Two decisions recur. First, for visual search, embed the product images with a multimodal model; embedding only titles and descriptions with the same model means every query is still matched against text. Reducing images to text first (captions or detected labels) and embedding that text discards the visual features that make two images look alike. Second, vector storage and index memory scale with the number of chunks times the number of dimensions. When storage dominates cost and a small accuracy loss is acceptable, re-embedding at a lower output dimension (256 instead of 1,024 is a 75% reduction per vector) keeps the same model and tokenization. Changing the dimension or the model always means re-embedding the whole corpus; tuning chunking and embeddings in detail belongs to Task 2.2.

Use Retrieval Augmented Generation (RAG) when the problem is knowledge, and fine-tuning when the problem is behaviour. RAG retrieves relevant passages at query time and gives them to the model, so it handles facts that change, answers that must cite a source, and per-user restrictions on which documents can be used. Fine-tuning changes the model's weights from examples, so it handles tone, fixed output structure, and narrow task conventions such as a proprietary labeling taxonomy.

The two combine. When an application needs current, citable facts and a behaviour that prompting alone fails to enforce, use retrieval for the facts and supervised fine-tuning on approved examples for the behaviour. Context copied into a static prompt by hand is not retrieval: it must be maintained manually, goes stale and carries no citations.

Pick the customization method by the data and signal you actually have, because each method needs a different input. The question to ask first is not "which method is best" but "do we have labeled pairs, a grader, a good teacher model, or only raw text?"

Limits matter as much as fit. Distillation cannot exceed its teacher: if no available model performs the task well, there is nothing good to distil, whereas any automatic way of grading outputs can drive reinforcement fine-tuning. Supervised fine-tuning needs examples of the desired outputs; reference documents on their own are material for continued pre-training or retrieval, not labeled pairs. Capacity options change throughput, not what the model has learned. Prompt engineering should always be tried first; the mechanics of prompting and of running training jobs are Task 2.2.

Parameter-efficient fine-tuning (PEFT) such as LoRA freezes the base model and trains small low-rank adapter matrices, while full-parameter fine-tuning updates every weight. LoRA therefore needs far less GPU memory and compute, trains faster, and produces an adapter that is a small fraction of the model's size; full fine-tuning can extract slightly more task performance but needs much more compute and produces a complete copy of the model per variant.

The hosting saving is what makes LoRA decisive for many low-traffic variants. Amazon SageMaker AI can serve multiple LoRA adapters as inference components on a single endpoint that holds one copy of the base model, so many variants share GPU capacity. The saving depends on keeping adapters separate at serving time: any design that turns each variant back into a full-size model gives it up, and folding all variants into a single model gives up independent updates.

Choose the RAG pattern by the shape of the questions and of the data: plain semantic vector search answers descriptive questions, but exact identifiers, multi-hop relationships, computed aggregates and access restrictions each need a different pattern. Diagnosing which kind of question fails tells you which pattern to add.

Hybrid search in Amazon Bedrock Knowledge Bases is supported only for Amazon RDS (Aurora PostgreSQL), Amazon OpenSearch Serverless and MongoDB vector stores that include a filterable text field; other stores fall back to semantic search. Raising the number of results returned does not fix ranking that is computed the wrong way, and a reranker can only reorder chunks that retrieval actually returned. Text retrieval over chunked copies of table data, or training on such copies, cannot compute reliable aggregates; only an engine that runs the query over the rows can. Amazon Kendra is a document search service: it retrieves documents (and can honour document access-control lists) rather than computing results. Tuning chunk size, reranking and retrieval settings in detail is covered in Tasks 2.2 and 3.1.

Choosing the vector store behind a knowledge base

Choose the vector store by the features you must have first (hybrid search, graph retrieval, SQL joins) and by cost and scale second. A cheaper store that lacks a mandatory feature is not an option, however attractive its price.

Read the scenario for the deciding constraint. When hybrid search is mandatory, pick a store that supports it even if query volume is low. When a collection is very large, rarely queried and purely semantic, S3 Vectors usually costs least. When vectors must be joined with records already held in Aurora PostgreSQL, add pgvector to that database rather than copying data to a new store. Creating and indexing the knowledge base itself is Task 3.2.

Trade-offs differ for training and for inference: for ML training, reduce time and cost by starting from pre-trained weights, paying less for the same compute, and choosing the simplest model whose accuracy meets the requirement. Faster hardware or more instances finish sooner but rarely cost less, and a small accuracy gain seldom justifies a much more expensive model.

• Transfer learning: fine-tuning a pre-trained network (for example an image classifier from SageMaker JumpStart) on a modest labeled set is faster than training from random weights and overfits less, because the network already knows general features. Distributed training or Spot pricing changes how fast or at what price a from-scratch job runs, not how much work it needs. If the model must be exported (for example to edge devices), a managed service whose models are only served by that service, such as Amazon Rekognition Custom Labels, does not qualify.
• Managed Spot training: runs SageMaker AI training on spare capacity at a large discount. Set a maximum wait time longer than the maximum run time so interruptions are tolerated, and save checkpoints to Amazon S3 so the training script resumes from the latest checkpoint instead of restarting. Spot without checkpoints risks losing hours of progress. Use it when the deadline has slack.
• Warm pools keep training instances provisioned between jobs to cut start-up latency; you pay for the retained instances, so they reduce waiting, not cost.
• Right-sizing: a larger instance may finish sooner but is billed at a higher rate; measure before assuming it is cheaper.

For foundation model inference, the levers are different and are covered below. Techniques inside the training job, such as early stopping, distributed training and hyperparameter tuning, belong to Task 2.2.

Foundation model performance, latency and cost

Lower FM cost and latency by matching each workload to the right lever: a smaller or distilled model where quality allows, prompt caching for repeated prompt prefixes, batch inference for non-interactive bulk jobs, Provisioned Throughput for steady high load, and cross-Region inference or routing for throughput and mixed difficulty. Each lever solves one problem and creates a constraint, so read the scenario's constraint before choosing.

Retrieved context counts as input tokens exactly like typed prompt text, so relocating repeated text does not shrink a request; caching lowers the cost of a repeated prefix instead. Moving all traffic to a cheaper model trades away quality on the hard requests, whereas routing and distillation aim only at the share a smaller model can handle.

Use Amazon Textract for documents, Amazon Rekognition for images and video, Amazon Transcribe for speech, and Amazon Comprehend for text; each offers pre-trained operations for common needs and customization only where the built-in model cannot know your vocabulary or labels.

Amazon Textract

Use AnalyzeExpense when invoices or receipts arrive in many layouts, because generic FORMS output keeps each layout's own labels and TABLES gives line items without normalized header fields. Mixed document sets combine operations, each on the documents it is built for: AnalyzeDocument for general forms and tables, AnalyzeID for identity documents. Rekognition text detection reads text in photos and video frames but returns no document structure or identity fields.

Amazon Rekognition

DetectModerationLabels returns a hierarchy of unsafe-content labels (explicit, violent and so on) with confidence scores, which can drive automatic blocking or send borderline images to human review. DetectLabels identifies general objects and scenes and is not a moderation model. Custom Labels trains a model on your own labeled images for business-specific objects; it needs labeled data, its models are served by Rekognition, and AWS does not position it for unsafe-content detection. To tune moderation to your own policy, Rekognition Custom Moderation trains an adapter on your annotated images that improves DetectModerationLabels accuracy.

Amazon Transcribe and Amazon Comprehend

Transcribe converts audio to text. PII redaction removes sensitive data such as card and Social Security numbers from transcripts, and a custom vocabulary improves recognition of product names and jargon. Comprehend works only on text, so audio must be transcribed first. Its pre-trained APIs detect sentiment (positive, negative, neutral, mixed), built-in entities, PII and key phrases; asynchronous jobs process large document collections, and per-document size limits differ by API. When the built-in entity types do not cover your terms, train a custom entity recognizer (from an entity list or annotations) to extract mentions; a custom classifier is the near-twin that assigns a label to a whole document instead of extracting mentions.

Some older purpose-built services are closed to new customers (for example Amazon Forecast); for a new forecasting workload use SageMaker AI, such as the DeepAR algorithm.