AI agents vs workflows: which one to build

The word "agent" covers two different designs, and the choice between them affects cost, latency, and reliability. Anthropic's guide to building agents separates them by who decides the next step. In a workflow, code decides the order: LLMs and tools follow predefined code paths. In an agent, the LLM directs its own process and tool use (Anthropic, 2024). Many business tasks fit the first design. The second suits tasks where the steps depend on what the system finds.

Start with the simplest thing that could work

The same guide recommends finding the simplest solution possible and adding complexity only when needed, which might mean not building an agentic system at all. It notes that agentic systems often trade latency and cost for better task performance, and that for many applications a single model call with retrieval and in-context examples is usually enough. It also reports that the most successful implementations its authors saw used simple, composable patterns rather than complex frameworks.

That gives a ladder with three rungs:

1. One model call, with retrieved documents and a few examples in the prompt.
2. A workflow: several model calls and tool calls, in an order you wrote down.
3. An agent: the model chooses the steps and the tools while it works.

Climb a rung only when testing on your own examples shows that the rung below fails. Our post on RAG vs fine-tuning describes how to sort those failures.

What a workflow looks like

Take a support team that wants faster replies. A workflow for it might run these steps in a fixed order: classify the incoming ticket, retrieve the matching policy passages, draft a reply that cites them, and send the draft to a person for approval. The steps never change. Each one can be tested on its own, and when a reply is wrong you can see which step failed.

This is an illustration, not a client result. The path is known before the system runs, so a workflow gives predictable behavior at a predictable cost.

What an agent adds

An agent is worth its complexity when the steps cannot be known in advance. The ReAct paper describes the pattern: the model produces reasoning steps and actions in an interleaved way, so the reasoning helps it track and update its plan, and the actions let it reach outside sources such as knowledge bases or environments to gather information (Yao et al., 2022). A research assistant that picks its next search based on what the last one returned is a fair example. So is a coding assistant that runs the tests and reads the errors before choosing its next edit.

The price is variability. Each run can take a different path, use a different number of steps, and cost a different amount.

A middle design works for many tasks: a workflow with one agentic step. The fixed steps handle the parts you can write down, and the model gets freedom only at the point where the path depends on what it finds.

The reliability problem

An agent also has to be consistent, and consistency is harder to get than a good demo. The tau-bench benchmark tests agents in conversations with a simulated user, using domain tools and policy rules. Its authors report that even the state-of-the-art function-calling agents they tested, including gpt-4o, succeeded on fewer than half of the tasks. The agents were also inconsistent: in the retail domain, success on all eight of eight repeated trials of the same task fell below 25% (Yao et al., 2024).

The paper tested models available in 2024, so read it for its method. Run each test case several times, and count a case as passed only when it passes every time. One good run proves little.

A decision test

• Can you write the steps down? If yes, build a workflow.
• Does the number of steps vary with what the system finds? If yes, an agent may be justified.
• Can a wrong action be undone? An email that has been sent or a payment that has moved cannot be recalled. Irreversible actions need a person in the loop, whatever the design.
• What are the cost and latency budgets? An agent that takes twenty model calls to finish a task needs a budget that allows for them. Our post on running costs shows how to estimate that.
• Who owns the outcome? A system that acts by itself needs a named owner who reviews what it did.

If you build an agent

A few habits keep the risk down. Give it the fewest tools that let it do the job, and start with read-only tools. Require confirmation before any action that cannot be undone. Cap the number of steps. Log every action along with the reason the agent gave for it.

Anthropic's guide lists three principles for agent design: keep the design simple, show the agent's planning steps so people can see what it is doing, and design the tool interfaces with care. Then test the agent the way the previous section describes, with repeated runs on a fixed evaluation set. Our post on evaluating a RAG system shows how to build one.

Where this leaves you

Write down the task as a list of steps. If you can complete the list, you have a workflow. If you cannot, mark where the steps depend on what the system finds along the way. Those points are where an agent might justify itself. Our AI agent development page says when that service is the wrong choice, and our process page describes how discovery tests the question before any build begins.