In the previous post I argued that most AI features should start as simple workflows. This one is about the cases where that genuinely isn't enough: tasks big or open-ended enough that several agents working together beat one agent working alone.

They do exist. But a multi-agent system is a distributed system made of components that occasionally improvise, so the engineering has to be deliberate.

Why multiple agents at all?

The best public evidence I know comes from Anthropic's write-up on how they built their multi-agent research system. A lead agent plans the research and spins up subagents that search in parallel, each with its own context window. On their internal research evaluation, the multi-agent setup outperformed a single agent by 90.2%.

The same article is honest about the cost: multi-agent systems used about 15× more tokens than ordinary chat interactions. Their analysis found that token usage alone explained 80% of the performance variance on a browsing benchmark. In other words, much of the gain comes from spending more compute, spread across agents that can each focus.

That gives a simple rule: multi-agent is worth it when the task is valuable, broad and parallelizable. It's a poor fit for tightly coupled work where every agent needs the same context, such as most single-file coding tasks.

Pattern 1: one orchestrator, many specialists

The architecture I use most is a supervisor (the orchestrator) that owns the plan and delegates to specialist agents, each with a narrow role and a small, well-described toolset:

• a retrieval agent that searches documents and returns cited snippets,
• an analysis agent that works only from what retrieval provides,
• a writer agent that turns findings into the final response,
• and sometimes a critic that checks the result against the original question.

Specialists stay simpler to prompt, easier to evaluate in isolation, and cheaper, because they can often run on a smaller model while the orchestrator uses a stronger one.

Pattern 2: explicit, typed state

The biggest source of multi-agent bugs I've seen is implicit state: agents passing loose text to each other and hoping the next one interprets it correctly.

Instead, give the system a typed state object that every node reads and writes, such as the plan, collected sources, intermediate findings, and a step counter. Graph frameworks like LangGraph are built around exactly this: nodes are agents or functions, edges are transitions, and the state is checkpointed between steps.

Checkpointing pays for itself immediately. You can resume a long run after a failure, replay a bad run to debug it, and pause for a human decision without losing work.

Pattern 3: handoffs with a contract

When one agent hands work to another, treat it like an API call between services. Define what the receiver gets, what it must return, and what "done" means.

The OpenAI Agents SDK models this directly as handoffs, where one agent delegates control to another with a defined input. Whatever framework you use, the principle is the same: vague delegation produces duplicated and missing work. A good task description states the objective, the expected output format, which tools to use, and clear boundaries ("don't re-search sources X and Y; they're already covered").

Pattern 4: guardrails at every boundary

Every place an agent touches the outside world is an attack surface and a failure point. The OWASP Top 10 for LLM Applications is the checklist I start from, and prompt injection is the one that bites multi-agent systems hardest: a malicious instruction hidden in a retrieved web page can travel from the retrieval agent all the way to an agent with write access.

What helps in practice:

• Least privilege: each agent gets only the tools its role needs. The retrieval agent can't send email.
• Treat tool output as data, never as instructions, and say so explicitly in prompts.
• Human approval for irreversible actions: payments, deletions, outbound messages. LangGraph's human-in-the-loop interrupts make this a pause, not a rewrite.
• Hard limits: maximum steps, maximum tokens, maximum wall-clock time per run.

Pattern 5: evaluate the system, not just the prompts

A change to one agent's prompt can quietly break another agent downstream. You need end-to-end evaluation on a fixed set of realistic tasks, run on every change.

For retrieval-heavy systems, Ragas gives you metrics like faithfulness and context precision out of the box. Pair it with a small set of LLM-as-judge rubrics for the final answer, and a handful of human-graded examples to check the judge itself. Start with twenty good test cases rather than waiting for two thousand.

Pattern 6: observability from day one

When a multi-agent run gives a bad answer, the question is always which agent, which step, which input? Without tracing you're guessing.

Emit a trace per run with a span per agent step: inputs, outputs, tool calls, tokens and latency. OpenTelemetry is a solid vendor-neutral base, and most LLM observability tools speak it. Two dashboards earn their keep immediately: cost per successful task, and where runs fail or loop.

A production checklist

Before a multi-agent feature ships, I want to be able to answer yes to all of these:

1. Is the task broad and valuable enough to justify roughly an order of magnitude more tokens?
2. Is shared state typed and checkpointed?
3. Does every handoff have a clear contract?
4. Does each agent have least-privilege tools, with approval on irreversible actions?
5. Are there hard limits on steps, tokens and time?
6. Is there an end-to-end eval set that runs on every change?
7. Can I trace any bad answer back to the step that caused it?

Multi-agent systems reward this discipline. Built carefully, they handle research and analysis tasks that a single agent simply can't. Built casually, they become very expensive ways to be confidently wrong.

I build these systems for a living. If you're weighing one up, take a look at my projects or send me a message.