What are AI agents? How they differ from a chatbot

How agents differ from a plain chatbot

A standard interaction with a large language model, covered in our LLM explainer, is a single round trip: you send a prompt, the model generates a response from its training and whatever context you gave it, and the exchange ends there. The model has no way to check a fact, run a calculation, or make a change outside the conversation — it can only produce text.

An agent wraps that same underlying model in a loop that lets it do more than talk. Instead of stopping after one response, the system lets the model decide on an action — search the web, run a piece of code, call an API — actually carries that action out, feeds the result back to the model, and lets it decide the next step. The conversation becomes a working process rather than a single exchange.

The core ideas, conceptually

Strip away the branding and most agent systems share the same handful of building blocks:

• A model as the reasoning engine — the language model decides what to do next, in plain language, based on the goal and everything that has happened so far.
• Tools — defined actions the agent is allowed to invoke, such as a search function, a code interpreter, or a call to an external service, each with a description the model uses to decide when it is relevant.
• Memory — a record of what has happened in the current task (and sometimes across tasks), so the agent does not lose track of earlier steps or results.
• A plan–act–observe loop — the agent proposes a step, carries it out, observes the outcome, and uses that outcome to decide the next step, repeating until the goal is met or it gives up.

What agents are used for

The pattern shows up wherever a task needs more than one step to complete. Research assistants use agents to search multiple sources, cross-check facts and assemble a summary rather than answering from memory alone. Coding assistants use them to read a codebase, write and run code, see whether tests pass, and iterate rather than producing a single untested guess. Customer-support automation uses agents to look up an account, check a policy, and take an action like issuing a refund within defined limits. Workflow automation more broadly uses agents to chain together steps across different systems that would otherwise need a person to move information manually from one to the other.

The honest limits

Agents are genuinely useful and genuinely over-hyped at the same time, and both things are worth saying plainly. The core risk is that errors compound: a single wrong step early in a multi-step task can send everything after it in the wrong direction, and the longer the task, the more chances there are for that to happen. The model can still hallucinate — inventing a fact or misreading a tool’s result — and an agent that acts on a hallucination does something, not just says something, which raises the stakes considerably.

Cost and reliability are real concerns too: a multi-step agent typically makes many more model calls than a single prompt, and a task is only as reliable as its least reliable step. None of this means agents should be avoided — it means they need guardrails: clear limits on what actions they can take unsupervised, human review at meaningful checkpoints, and the same responsible-AI thinking that applies to any AI system, only more so because agents act rather than merely answer.

How agents relate to RAG and tool use

Agents are often built on top of two ideas covered elsewhere on this blog. Many agents use retrieval-augmented generation, explained in our RAG article, so that a step in their plan can be "look this up in our documents" rather than relying on the model’s training alone. And the mechanism that lets a model reliably say "call this specific function with these specific arguments" — usually called function calling or tool use — is what makes the tools described above something a model can actually invoke rather than merely describe.

The cloud angle, and agents in cert study

All three major cloud providers now offer tooling for building agents — for example, Amazon Bedrock includes agent-building capability that handles the orchestration loop, tool integration and memory described above so teams do not have to build that scaffolding themselves. Azure and Google Cloud offer comparable agent-building services within their own AI platforms. The specifics of any one product change quickly, so the durable knowledge is the pattern itself, not a particular vendor’s feature set.

Agentic AI is an emerging area that shows up as a topic in AI-focused certification study, including the AWS Certified AI Practitioner (AIF-C01), which expects a conceptual grasp of what agents are and how they extend a plain model — exactly the ground this article covers. For anyone building agents in practice rather than just being examined on the concept, the AWS Certified Developer – Associate is the more hands-on complement.