Subscribe
educational
5 min read

What Is an AI Agent? How Language Models Run Tools in a Loop

For three years, "AI agent" meant whatever a vendor's pitch deck needed it to mean. Now the people who build these systems have largely converged on one definition, and it is disappointingly small: an agent is a language model using tools in a loop to reach a goal. No mystery substance, no ghost in the machine. A loop. This post explains what that loop is, where it came from, and what is actually inside it. Chatbot versus agent: one call or many A chatbot answers one question per call. You s

What Is an AI Agent? How Language Models Run Tools in a Loop

For three years, "AI agent" meant whatever a vendor's pitch deck needed it to mean. Now the people who build these systems have largely converged on one definition, and it is disappointingly small: an agent is a language model using tools in a loop to reach a goal. No mystery substance, no ghost in the machine. A loop.

This post explains what that loop is, where it came from, and what is actually inside it.

Chatbot versus agent: one call or many

A chatbot answers one question per call. You send text, the model returns text, done. An agent is different in exactly one structural way: the model's output can trigger an action, the result of that action is fed back to the model, and the model decides what to do next.

Anthropic's engineering description, which the industry has broadly adopted, puts it this way: agents are "typically just LLMs using tools based on environmental feedback in a loop." In a later post on context engineering, the company narrowed it even further: "LLMs autonomously using tools in a loop." Simon Willison's shorthand adds the piece the others leave implicit: an agent runs tools in a loop to achieve a goal, which means there is a stopping condition. The loop ends when the model says the task is finished, when it gives up, or when a budget runs out.

The distinction matters because it locates the autonomy. Nothing in the loop is magic. The model is still predicting tokens. What changed is that those token predictions now select actions, and the consequences of those actions become the model's next input.

What is inside the loop

In June 2023, OpenAI's safety lead Lilian Weng published the survey that became the standard map of agent architecture. Her formula: agent = LLM + memory + planning + tool use. The LLM is the controller. The other three are scaffolding around it.

The agent loop: model, tools, observation, and a stopping condition

Tools are functions the agent can call: a search engine, a calculator, a code interpreter, a database, an API. They matter because the model itself cannot browse, compute reliably, or act on the world. Text generation is all it has natively, so tools are how an agent touches anything outside its context window.

Memory comes in two flavors. Short-term memory is the conversation itself: every tool call and result is appended to the context, so previous steps become what the agent "remembers" about the current task. Long-term memory is anything outside the window, usually files or databases the agent writes to and reads back. Anthropic's memory tool, shipped in beta with the Claude Sonnet 4.5 launch in September 2025, formalizes the file-based approach: the agent maintains notes on disk and consults them instead of keeping everything in context.

Planning is the least physical component. It is the model decomposing a goal into steps and adjusting when steps fail, which in practice mostly means the loop gets another turn. Weng's post catalogued the techniques of that moment, chain-of-thought, tree search, reflection passes, and flagged the enduring weakness: planning errors compound, so one wrong assumption early can cascade through everything after it.

How a single step works

The mechanics in production systems are boring and well-defined. OpenAI's function calling, introduced on June 13, 2023, set the template. A developer gives the model a list of tool definitions with JSON schemas. The model, fine-tuned for this, either answers normally or returns a JSON object naming a function and its arguments. The application runs the function. The result is inserted into the conversation, and the model is called again, now seeing the outcome. Repeat until the model produces a plain answer instead of a tool call.

Note who executes the function: not the model. The model proposes, the harness disposes. The agent loop is a conversation where some messages happen to be API responses.

A tool call leaves the model, executes externally, and the result flows back into the conversation

Where the loop came from

The pattern was established by two papers in 2022 and 2023.

ReAct (Yao et al., October 2022, published at ICLR 2023) showed a language model generating reasoning traces and tool actions in alternation: think, act, observe, think again. With a simple Wikipedia API, ReAct reduced hallucination and error propagation on question answering compared to reasoning alone, and on interactive benchmarks it beat imitation and reinforcement learning baselines by 33 and 10 absolute success rate points on ALFWorld and WebShop while using only a handful of prompted examples.

Toolformer (Schick et al., February 2023) attacked the problem from the training side: a model can learn which APIs to call, when, and with what arguments, self-supervised, from a handful of demonstrations per tool. Its 6.7B model gained large zero-shot improvements from calculator, search, translation and calendar calls. Its limitations were also instructive. It could not chain tools, using one call's output as another's input, because each call was generated independently. The loop had to be assembled separately.

Between 2022 and 2023 those pieces merged: the interleaved reason-act-observe cycle became a runtime loop, the API calls became schema-validated function calls, and the whole thing got a name.

Agents versus workflows: where the control flow lives

Anthropic draws the most useful boundary line in the space between "agentic systems" of all shapes. A workflow orchestrates models and tools through predefined code paths: fixed sequences, branching written by the developer. An agent lets the model itself decide what happens next, which tools to call and when to stop. Both are legitimate; they answer different questions. How much control is fixed in code versus decided by the model at runtime?

The same guidance applies to choosing between them. Anthropic's recommendation is to use the simplest solution that works, often a single augmented LLM call or a workflow, and to reach for open-ended agent autonomy only when flexibility and model-driven decisions justify the cost. Complexity buys capability and unreliability in the same transaction.

What the loop actually gets you

The GAIA benchmark (Mialon et al., November 2023) captured the starting line. Its 466 assistant questions, real-world tasks needing browsing, multimodality, and tool proficiency, were conceptually simple for people: human respondents scored 92%. GPT-4 equipped with plugins managed 15%. The gap was not knowledge but execution, the ability to string actions together robustly over many steps.

That gap has been closing, and coding assistants that edit repositories, run tests and iterate until they pass are the most convincing evidence: a model in a loop, checking its own work against an environment that pushes back. That turns out to be the whole trick, and the whole difficulty. Environments with ground truth (compilers, tests, servers) make agents trustworthy; fuzzy environments make them confidently wrong, in a loop, at scale.

If you take one thing away

An AI agent is a language model given tools and a feedback loop, pointed at a goal, and released until it stops. The model does the thinking; the harness does the executing; the context window is the memory; the loop is the planning. Every impressive agent demo and every embarrassing one is the same mechanism, run to different outcomes by different tools, goals, and stopping conditions.

Sources

educationalAI AgentsLLM