What are large language models? LLMs explained simply

The core idea: predicting the next word, at scale

Strip away the mystique and an LLM does one thing: given a passage of text, it predicts what is most likely to come next. Shown “the capital of France is”, it predicts “Paris”. Shown half an email, it predicts a plausible continuation. Generation is simply this prediction run in a loop — predict the next small piece of text, add it to the passage, predict again — until a full answer has been assembled one step at a time.

What makes this humble mechanism so powerful is scale. Trained across a vast sweep of human writing, predicting the next word well turns out to require absorbing an extraordinary amount along the way: grammar, facts, reasoning patterns, styles, conventions of every genre from legal contracts to source code. The model was never explicitly taught to summarise or translate; those abilities emerge because doing them is, in the end, a form of predicting appropriate text. That is the single most useful mental model to carry into any AI exam: an LLM is a next-word predictor so capable that prediction starts to look like understanding.

What “large” actually means

The “large” refers to two things at once. First, the training data: LLMs learn from text collections of enormous breadth — books, articles, websites, documentation, code. Second, the model itself: internally it consists of a vast number of adjustable values called parameters, which are tuned during training and collectively store everything the model has learned. More parameters give a model more capacity to capture subtle patterns, roughly as a bigger library can hold more books.

You will see specific parameter counts quoted in marketing and press coverage. Treat them loosely: the figures leap with every release, vendors do not always disclose them, and a bigger number does not straightforwardly mean a better model — training data quality and technique matter as much as raw size. For understanding the concept, and for certification purposes, “very many, tuned automatically during training” is the right level of detail.

Training versus inference: learning once, answering many times

An LLM’s life has two sharply different phases. Training is the learning phase: the model processes its huge text collection, gradually adjusting its parameters until its predictions are good. It happens once (plus occasional updates), takes serious time and computing power, and is carried out by the organisation building the model — not by you.

Inference is the using phase: the trained model, parameters now frozen, takes your prompt and generates a response. Every chat message you send is inference. The distinction matters practically. It explains why the model does not learn from your conversations by default — nothing about it changes at inference time. It explains the knowledge cutoff — the model knows nothing after its training data was gathered. And it explains the economics of the field: a few organisations train foundation models at great expense, and everyone else builds applications on top of them through inference, often via managed cloud services.

Tokens and context windows

Two pieces of vocabulary appear constantly around LLMs and are easier than they sound. A token is the unit of text a model actually reads and writes — typically a word or a piece of a word. “Certification” might be split into a few tokens; “the” is one. Models predict token by token, and usage of commercial models is commonly measured and billed in tokens.

The context window is the total amount of text — prompt plus response — the model can consider in one exchange. Think of it as working memory: everything inside the window can inform the answer; anything beyond it effectively does not exist for the model. This is why an assistant can seem to “forget” the start of a very long conversation, and why summarising a huge document may require breaking it into pieces. Window sizes vary by model and keep growing, so check a vendor’s documentation for current figures — the concept is the durable part.

The hard limits

Fluency is not understanding, and an honest account of LLMs has to say so plainly:

• No true comprehension — the model manipulates patterns in text; it has no beliefs, goals or model of the world, however convincing its prose.
• Hallucination — because output is plausible prediction rather than fact retrieval, a model can assert falsehoods fluently and confidently.
• Knowledge cutoff — it knows nothing after its training data was collected, and by default nothing about your private documents or systems.
• Context limits — only what fits in the context window can inform an answer; long inputs get truncated or must be chunked.
• Inherited bias — the model reflects the biases and gaps of the text it learned from, which is a central concern of responsible AI.

Where LLMs sit in certification study

LLMs are foundational content for the AWS Certified AI Practitioner (AIF-C01), which expects you to explain what a large language model is, how it relates to foundation models and generative AI, what tokens and context windows are, and — pointedly — what the limits are: hallucination, cutoff and bias all feature in its responsible-AI framing. Techniques for working around the limits, such as retrieval-augmented generation, appear there too; our separate explainer covers RAG in depth. The AWS Certified Cloud Practitioner touches this more lightly, expecting awareness of AWS’s AI services and what they are for.

Notice what the exams do not ask: which model is best this month, or how many parameters anything has. Those answers change weekly and certifications sensibly avoid them. The concepts in this article — prediction at scale, training versus inference, tokens, context, the honest limits — are the stable layer, and they are what both the exams and real-world judgement actually require.