How the agent loop works, step by step

Under the branding, every AI agent is the same small loop. You send a conversation to a model. It replies with either a final answer or a request to run a tool. If it asked for a tool, you run it, add the result to the conversation, and send everything back. You stop when the model says it's finished or when a limit you set says enough.

If you've read what an AI agent is, this is the next layer down. Once you can trace one turn by hand, agent frameworks stop looking mysterious, and most bugs become easy to spot.

The model has no memory between calls#

The first thing to internalize is that a model API is stateless. Nothing carries over from the previous request. If the model needs to know what happened three steps ago, you must send that history again.

Each call typically includes three things:

• A system prompt that describes the role, rules, and any standing context.
• Tool definitions: a name, a plain-language description, and a schema for the inputs each tool accepts.
• The message history: the user's goal, every earlier model reply, and every tool result so far.

This is why long agent runs get expensive. Step twenty resends everything from steps one through nineteen. It's also why the loop is really a loop over your code. The model doesn't remember anything; your program is the memory.

One full turn, traced#

Here's a run of a tiny agent with two tools, search_docs and read_file. The user asks: "Why does the export job time out?" This is the message history as it grows, shown as simplified JSON.

[
  {"role": "user", "content": "Why does the export job time out?"},

  {"role": "assistant", "content": [
    {"type": "text", "text": "I'll check the docs for timeout settings first."},
    {"type": "tool_use", "id": "toolu_01", "name": "search_docs",
     "input": {"query": "export job timeout"}}
  ]},

  {"role": "user", "content": [
    {"type": "tool_result", "tool_use_id": "toolu_01",
     "content": "exports/README.md: default timeout is 30s, set EXPORT_TIMEOUT to change"}
  ]},

  {"role": "assistant", "content": [
    {"type": "tool_use", "id": "toolu_02", "name": "read_file",
     "input": {"path": "config/exports.php"}}
  ]},

  {"role": "user", "content": [
    {"type": "tool_result", "tool_use_id": "toolu_02",
     "content": "'timeout' => 30, 'chunk_size' => 50000"}
  ]},

  {"role": "assistant", "content": [
    {"type": "text", "text": "The timeout is 30 seconds and chunks are 50,000 rows. Large exports likely exceed 30s. Raise EXPORT_TIMEOUT or lower chunk_size."}
  ]}
]

Read it top to bottom and the whole mechanism is visible.

The assistant turn contains a tool_use block. That's the model asking your code to do something. It does not run anything itself. Your program reads the request, executes the real function, and sends the output back as a tool_result in the next user message, tagged with the same id so the model can match answer to question.

The last assistant turn has only text and no tool request. That's the signal that the model considers the job done.

How the loop knows when to stop#

In most APIs the response carries a reason it stopped generating. With Anthropic's Messages API the field is stop_reason. The values you'll meet most often:

• tool_use: the model wants a tool run. Execute it and continue.
• end_turn: the model finished. Return the answer.
• max_tokens: the reply was cut off by your output limit. Usually a bug or a sign you need a higher cap.
• stop_sequence: a custom stop string was hit.

That gives you the core of the loop condition: continue while the reason is tool_use. But relying on the model alone to end things is a bad idea. Real loops add their own brakes:

1. A step limit, say 15 or 25 iterations, after which you stop and report what happened.
2. A token or cost budget that halts the run when it's spent.
3. A wall-clock timeout, since a hung tool shouldn't hang the agent.
4. A repeat detector that notices the same tool call with the same input three times in a row.

Think of these as fuses. You'll hit them rarely, and on the day you do, they're the difference between a confusing log and a surprising invoice.

Parallel tool calls#

Models can request several tools in one turn. Say the agent wants the weather in two cities. Instead of two round trips, the reply contains two tool_use blocks at once.

Your job is to run both, then send back one user message containing both results, each matched to its id. Running them concurrently is a nice speedup when the calls are independent, and it saves an entire model call.

The classic bug here is answering only one of the two. The API will reject the next request, because every tool_use needs a matching tool_result. If you see an error about missing tool results, look at your handling of multi-call turns first.

Errors are just more input#

A tool will fail sometimes. The file doesn't exist, the API returns a 500, the query is malformed. What should the loop do?

Don't crash and don't hide it. Return the failure to the model as a tool result, marked as an error where the API supports it:

{"type": "tool_result", "tool_use_id": "toolu_03",
 "is_error": true,
 "content": "FileNotFoundError: config/export.php (did you mean config/exports.php?)"}

A capable model reads that, corrects the path, and tries again. That self-correction is a big part of why agents feel different from scripts. A useful habit is writing error messages for the model as well as for humans. "Invalid input" gives it nothing. "Expected a date in YYYY-MM-DD format, got 'next Tuesday'" tells it exactly what to change.

Keep a limit on retries, though. An agent that can't fix a problem in three attempts will often invent a fourth, fifth, and sixth approach, each a little stranger than the last.

Context grows every step#

Each iteration appends at least two messages, the assistant's turn and the tool result. Tool results can be large: a full web page, a 2,000-line file, a big JSON blob. It's easy to watch a run go from a few thousand tokens to well over a hundred thousand in a handful of steps.

Three consequences follow.

Cost climbs faster than the step count. Every call re-reads the accumulated history, so the total tokens processed grow roughly with the square of the number of steps. Prompt caching softens this a lot, since the unchanged start of the conversation can be reused at a lower price.

Quality can drop. Long, cluttered contexts make it harder for the model to find what matters. Stuffing everything in isn't free even when it fits. That's the core argument of context engineering, which is really the craft of deciding what stays in the window.

You need a plan for trimming. Options include truncating oversized tool outputs, summarizing older turns, or moving detail into files the agent can re-read when it needs it. Pick one deliberately rather than waiting until you hit the context limit.

Where humans fit in#

The loop has a natural place for a person: the moment between "the model asked for a tool" and "you ran it." That's a checkpoint. You can auto-approve harmless reads, pause for a human on anything that writes, spends money, or sends a message, and refuse outright on things the agent should never do.

Doing this in code is simple. Before executing, look up the tool name in a policy table: allow, ask, or deny. It's some of the highest-value code in the whole system. If you're building anything that touches real data, read least privilege for AI agents before you wire up your first write tool.

What to log#

Debugging an agent means reading its transcript, so record it from day one. For every iteration, keep:

• the full request you sent (or a hash and the diff from the last one),
• the model's reply, including any text alongside tool calls,
• the tool name, inputs, and outputs, plus how long it took,
• token counts and the stop reason,
• a run ID so you can group steps and replay them.

When a run goes wrong, you'll open this log and read it like a story: what did it believe at step four, and why did step five follow from that? Without it you're guessing.

Common bugs, quickly#

A short list of things that account for a surprising share of "my agent is broken" reports:

• Forgetting to append the assistant message before the tool result. The API expects the tool_use turn to be in the history.
• Mismatched IDs. The tool_use_id in your result must equal the id from the request.
• Returning huge tool output. One 300 KB page can swamp everything else in the window. Truncate and say you truncated.
• Vague tool descriptions. The model picks tools from their descriptions, so a fuzzy description means fuzzy choices.
• No step limit. Fine until the first loop that never converges.
• Treating any text as "done." The model often writes a sentence and a tool call in the same turn. Check the stop reason, not just whether text exists.

Long-running tools and streaming#

Not every tool returns in a hundred milliseconds. A test suite might take two minutes, a web crawl longer. Two patterns keep the loop responsive.

The first is to let slow tools run in the background and return a handle immediately. The tool says "started job 88," and a second tool, check_job, reports progress. The model can do other useful work in between, or simply wait by polling a sensible number of times. This also gives you a natural place to enforce timeouts.

The second is streaming. Most model APIs can stream the reply token by token, which lets a user watch the agent think instead of staring at a spinner. Tool requests arrive as the stream completes, so you still execute them at the same point in the loop. Streaming changes what the user sees, not the structure of the run.

One caution: if a tool streams progress logs back into the context as it works, you'll fill the window fast. Summarize while running and keep the full log in a file the agent can open if it needs it.

A minimal implementation checklist#

If you're writing the loop yourself, this is the shortest list that keeps you out of trouble:

1. Keep the message history in your own variable and append to it in the right order.
2. After every model call, append the assistant message exactly as returned, tool blocks included.
3. For each tool request, run the tool inside a try/except and always produce a result, even on failure.
4. Send all results for that turn back in one user message.
5. Stop on end_turn, and stop on your own limits.
6. Log everything you can afford to log.

That's it. Everything else people add on top, whether planning steps, reflection passes, or sub-agents, is a variation on those six moves. If the basics are sloppy, the fancy parts won't rescue them.

What frameworks add, and what they hide#

Agent frameworks and SDKs wrap this loop so you don't write it by hand. They're a fair choice once you understand what they're doing, and they typically add tool registration, retries, tracing, and streaming helpers. Some also bundle memory, planning, or multi-agent handoffs.

The trade is visibility. When a framework runs the loop, the messages sent to the model are assembled somewhere you can't easily see. If a run misbehaves, your first task is to find the exact request that went out. A framework that lets you print or trace the final prompt is worth more than one with ten extra features. If you can't inspect it, be careful about depending on it for anything that matters.

A sensible path is to write the raw loop first for a small project, feel how little there is to it, and only then pick up a library for the conveniences you actually miss.

Try it yourself#

The fastest way to make this stick is to write the loop once with no framework. About forty lines of Python will do it, and building your first agent with the Claude API gives you a version to start from. Once you've seen the loop with your own eyes, frameworks and SDKs become conveniences instead of magic, and you'll know exactly which piece to blame when something misbehaves.