Agentic workflows 101: when to let a model run the loop

Three shapes of LLM system

Most confusion about agents comes from using one word for three different things. It helps to name them.

A pipeline is a fixed sequence of steps, some of which call a model. Extract the invoice fields, validate them, post to the ledger. The code decides the order; the model fills in a step. This is how most document processing works, and it is the right shape for anything repetitive and well understood.

A workflow with LLM decisions adds branching. The model classifies a ticket, and the code routes it. The model still does not choose what happens next beyond a menu you defined.

An agent inverts control. You give the model a goal and a set of tools, and it decides which tools to call, in what order, how many times, until it judges the goal met. The code's job becomes enforcing limits rather than dictating steps.

The further right you go on that spectrum, the more the system can handle situations you did not anticipate, and the less you can predict what it will do. That trade is the whole subject of this guide.

The loop

Every agent, whatever the framework, is this loop. The model is given the goal, the conversation so far and the list of tools. It produces either a final answer or a request to call a tool. The code executes the tool, appends the result and calls the model again. It stops when the model answers, when a step or spend budget is exhausted, or when a check fails.

The quality of an agent is decided far more by what surrounds the loop than by the model inside it: the tools, the stopping rules, the permissions and the trace.

Tool design decides everything

A tool is a function the model can call, described in words. The description is the model's only documentation, so write it for a capable new colleague: what it does, when to use it, what it returns and what it must not be used for.

• Narrow beats general. get_order_status(order_id) is called correctly; query_database(sql) is called creatively.
• Return only what the next step needs. Large results flood the context window and bury the useful part.
• Make errors recoverable. "No order with that id; the customer may have used a reference number" lets the model try again sensibly.
• Separate reads from writes. Reads can be free; writes need permission, confirmation or both.
• Standardise the plumbing. If more than one assistant needs the same tools, serve them through MCP, as described in MCP explained.

A useful test: could a careful human do the task with only these tools and these descriptions? If not, the agent cannot either. More on this in tool use.

Stopping conditions and budgets

The model should never be the only thing deciding when to stop. Set three limits in code: a maximum number of steps per run, a maximum token or spend budget per run, and a daily cap per user or tenant. Add a wall-clock timeout. When any limit trips, the agent returns what it has, with a clear statement that it stopped early, and the run is flagged for review.

Also define what "done" means before you start. "Reply to the customer" is not a stopping condition; "a draft reply exists, cites the relevant policy and has been queued for approval" is. Vague goals are the single biggest cause of agents that wander.

Where the human sits

Autonomy is a dial, not a switch. The patterns below are listed in order of increasing independence; most production agents live in the middle.

Move one step along the dial only when the evaluation numbers say the previous step is reliable. This is the approach behind our customer support AI and sales copilot solutions, which both start at "suggest".

When an agent beats a pipeline

Agents are worth their unpredictability when the path to the goal genuinely varies case by case and the tools to discover it exist. They are not worth it when the path is known.

A useful compromise is a pipeline whose steps are small agents: a fixed outer structure, bounded autonomy inside each step. It gives you predictability where you need it and flexibility where it pays.

Failure modes to design out

Every one of these should be a test case. Our AI red teaming service builds adversarial suites for exactly these behaviours.

Evaluating an agent

You cannot evaluate an agent by reading a few transcripts. Build a set of tasks with known good outcomes, run the agent against them repeatedly, and measure task success, steps taken, cost per run and the rate of unsafe actions attempted. Record the full trace of every run in production and sample them for review. Treat a drop in task success or a rise in steps as a regression, exactly as you would a failing unit test. The method is laid out in evaluating LLM outputs; the guardrails entry covers the runtime side.

A sensible first agent

Pick an internal, read-mostly task that people find tedious and that touches two or three systems: preparing a customer summary before a call, triaging an inbox, investigating a failed job. Give it three or four narrow tools, a step budget of a dozen, and the "suggest" pattern. Run it on a hundred real cases, review the traces, fix the tools. Only then consider a write tool with approval. This is the path our AI agents and automation engagements follow, and it rarely takes more than a few weeks to reach something people use daily.

Key takeaways

• Pipeline, workflow and agent are different shapes; choose by how much the path varies.
• The loop is simple; tools, stopping rules, permissions and traces decide the outcome.
• Write narrow tools with descriptions a new colleague could follow.
• Enforce step, spend and time budgets in code, never in the prompt.
• Start at "suggest" and move along the autonomy dial only when the numbers justify it.
• Treat tool results as untrusted; a web page can instruct your agent.
• Evaluate on repeated task runs and keep every trace.

Related services