Training and Fine-Tuning Foundation Models

Training a foundation model means adjusting its internal parameters (its weights) so it captures patterns in data. The exam asks you to recognize four key elements of training, what data each one learns from, and where each sits on an effort-and-cost spectrum. You almost never pre-train your own model; instead you take a pre-trained FM from a provider and customize it.

Why the costs line up this way. Training cost grows with how much data the model processes and how large the model is. Pre-training starts from nothing and reads an enormous corpus. Continuous pre-training starts from a capable model, but it still processes far larger corpora than fine-tuning does (millions of documents rather than thousands of examples), so it sits above fine-tuning. Fine-tuning reuses everything the model already knows and only needs a focused labeled dataset, so it is the cheapest of the three. Match the goal to the element: broad new knowledge points to pre-training, deeper domain knowledge from raw text points to continuous pre-training, task-specific behavior points to fine-tuning, and a cheaper, faster model points to distillation.

Pre-training is where a foundation model learns language, reasoning, and general knowledge from scratch. It consumes enormous unlabeled datasets and large amounts of compute, so it is the most expensive and time-consuming step. Providers do this for you; on the job you consume the result rather than repeat it.

Why no labels are needed. Pre-training is self-supervised: the model repeatedly predicts the next token (or a masked token) in ordinary text, and the actual next token in that text is the correct answer. The text supplies its own labels, so nobody has to write target outputs, which is what makes training on a massive corpus practical.

Continuous pre-training, which Amazon Bedrock and the AWS documentation call continued pre-training, takes an already pre-trained model and keeps training it the same self-supervised way on additional unlabeled text from your domain. Use it when a model needs to absorb the vocabulary and concepts of a specialized field, such as medical or legal text, and you have plenty of raw documents but no labeled task examples. It deepens subject knowledge without starting over, so it costs less than pre-training but more than fine-tuning.

Used together, in order. Continuous pre-training and fine-tuning solve different problems, so they are often combined in sequence: first continue pre-training on unlabeled domain documents so the model understands the field's terms, then fine-tune on labeled pairs so it performs the task in the required way. Reversing the order wastes effort, because broad training on raw text afterwards can dilute the task behavior you just taught. If you have only labeled pairs and a moderate vocabulary gap, fine-tuning alone may be enough and costs less.

Fine-tuning takes a pre-trained foundation model and trains it further on a smaller set of labeled examples so it performs your specific task or matches your style. Because this is supervised training (each input is paired with the output you want), it is often called supervised fine-tuning. It updates the model's weights, so it changes the model's behavior on every future request, not just a single response.

You choose fine-tuning when prompt engineering alone cannot get consistent results and the change you need is about behavior rather than fresh facts. Typical goals include matching a brand's tone, following a fixed output format, classifying support tickets into your categories, or answering in a domain's style. You supply pairs of inputs and desired outputs, and training nudges the weights toward producing those outputs. On AWS you can fine-tune supported models in Amazon Bedrock (custom models) or in Amazon SageMaker AI, for example through SageMaker JumpStart.

Fine-tuning costs more than prompting and requires good labeled data, but far less than pre-training because you start from a capable model. It also carries a maintenance cost: what the model learns is frozen at training time, so if the facts change you would have to fine-tune again. That is why facts that change often belong in retrieval, while stable behavior belongs in fine-tuning.

The exam lists several fine-tuning methods. They overlap, so focus on the goal each one serves and the data it uses.

Instruction tuning improves general instruction-following rather than one narrow task. Domain adaptation specializes a general model for a subject area. It builds on an existing pre-trained model, using labeled examples from the field and, where raw documents are plentiful, continued pre-training first; building a new domain model from scratch would be pre-training, the most expensive option. Transfer learning is the principle behind all fine-tuning: you reuse the general patterns learned during pre-training instead of starting over, which is why a few thousand examples can be enough where training from scratch would need vastly more.

Model distillation trains a smaller "student" model to reproduce the behavior of a large, capable "teacher" model. You send prompts to the teacher, and its responses become the training targets for the student. The result is a compact model that keeps much of the teacher's quality while being cheaper and faster to run.

It needs little labeled data. Because the teacher generates the target outputs, you only need representative prompts, such as real production requests, not human-written answers. That makes distillation a good fit when you have plenty of unlabeled prompts but few labeled examples. Amazon Bedrock Model Distillation is the managed option: you choose a teacher and a smaller student model, supply prompts (or use your stored invocation logs), Bedrock has the teacher generate responses, and it fine-tunes the student on them.

Choose distillation when a big model performs well but costs too much or responds too slowly for production, especially at high request volumes, or when it must run on constrained hardware such as edge devices. Because inference cost scales with model size, a distilled model can cut cost per request and latency significantly. The trade-off is that the student may lose some capability, so you evaluate whether it still meets your quality bar. Remember the roles: pre-training and continuous pre-training add knowledge, fine-tuning adapts behavior, and distillation reduces size and cost while preserving as much performance as possible.

A fine-tuned model is only as good as its training data, and the exam expects you to know five preparation activities.

Size versus quality. A smaller set of reviewed, correct examples usually produces a better model than a much larger set full of duplicates and errors, because the model learns the mistakes too. Removing exact duplicates alone does not fix examples that are wrong or off-topic.

Consistent labeling. The model learns exactly what you show it. If the same kind of input is paired with different target outputs, whether because reviewers disagree or because each example deliberately carries several alternative answers, the training signal is mixed and the model learns no clear behavior. Write labeling guidelines, check agreement between labelers, and fix disagreements before training.

Representativeness. If the data under-represents a group, language, or scenario, the model performs worse for it, which is a form of bias. Fix the cause in the data: add genuine, reviewed examples of the missing group until its share roughly matches what the model will see in live traffic. Two tempting fixes do not work. Copying the few existing examples adds volume but not new coverage, and the model tends to overfit to those repeated items. Overshooting, so the minority group far exceeds its real share, simply creates a new skew in the other direction.

Reinforcement learning from human feedback (RLHF) aligns a model with human preferences. Human reviewers compare several of the model's own responses to the same prompt and rank them (or pick the better of two). Those rankings train a reward model that scores how much people would prefer a response, and the foundation model is then optimized to produce responses that score higher.

RLHF matters because a pre-trained model predicts likely text, which is not the same as being helpful, honest, and safe. Learning from human judgments makes the model better at following the intent of a request and at declining harmful requests. It does not add new knowledge from documents and does not make the model smaller.

RLHF versus supervised fine-tuning. These two are easy to confuse because both use human input. The difference is what the humans produce.

In practice the two are layered: a model is usually instruction-tuned first, then refined with RLHF. In data-preparation terms, RLHF needs a different kind of data from fine-tuning: human preference judgments rather than labeled pairs.

Only training changes the model. Fine-tuning, continuous pre-training, distillation, and RLHF all update weights; Retrieval Augmented Generation (RAG), prompt engineering, and in-context learning act only at request time.

In-context learning is not fine-tuning. When you place examples in a prompt, the model uses them for that request only; its weights do not change, and the examples must be sent again every time. Fine-tuning on the same examples bakes the behavior into the model.

Combining approaches. Real applications often combine them, and the dividing line is how often something changes. Facts that change often belong in retrieval, because updating a document store is cheap while retraining is not. Stable format or behavior that prompting cannot enforce reliably belongs in a fine-tune, which you train once and reuse. Putting volatile facts into the weights instead would mean retraining whenever they change. Task 3.1 covers the cost trade-offs of these customization choices in more depth.