An incident has been resolved, and you want an agent to turn the notes into a briefing. The timeline, root cause and mitigation each need different evidence, so dividing the work among specialist agents sounds useful. You might let a supervisor assign the sections, write them in parallel, or ask several agents to compare drafts. Each arrangement changes how much context gets read and how long the briefing takes to finish.
Dividing the briefing improves coverage here, but each call adds context cost. The scripted single agent gets one of six sections right for $0.004. All four decomposing patterns recover four sections. Fan-out costs about $0.012, against debate's $0.033. We build their call graphs and count context and dependencies to explain the difference.
Anthropic's December 2024 "Building Effective Agents" recommends choosing the simplest composition that works and using several agents when a task decomposes cleanly. AutoGen, MetaGPT and the multiagent-debate papers describe different compositions. We instrument the calls to compare their context costs, scheduling dependencies and resulting answers on one task.
The measurement is cheap at small scale. We compare four call patterns: a sequential chain, a supervisor delegating to specialists, a parallel fan-out with a reduce step, and a debate panel with a judge. This post builds all four over one job against one scripted model, counting every token and every dependency between calls. You will see why the fan-out's speed-up is 1.3 times rather than six, watch a rate limit turn it into a slowdown, and find the whole table beaten on accuracy and cost at once by one agent with a better retriever. Everything runs in about a second with no API key.
Let's Build It
Open this in Google Colab and run it top to bottom.
Here are the five schedules against one clock. Each bar is a model call, drawn where the scheduler runs it. Watch the green synthesis bars, and watch how long the debate lane spends filling before it can start one.

The job is one incident briefing with six sections, written from a corpus of eighteen documents. Six of those documents carry the facts a section needs. The other twelve are the ordinary contents of a company wiki, including two earlier incidents whose postmortems use the same vocabulary.
SECTIONS = ["timeline", "root_cause", "mitigation",
"blast_radius", "comms", "regression_risk"]
Each pattern returns the same object: a list of Call records forming a dependency graph. That is the whole instrument.
@dataclass
class Call:
call_id: str
role: str # single, worker, router, drafter, judge, reduce
deps: tuple[str, ...] # the calls that must finish first
prompt_tokens: int
output_tokens: int
Cost is a sum over the graph. Wall-clock is the graph's critical path, which is a scheduler question and not a token question.
for pattern in ("single", "sequential", "supervisor", "parallel", "debate"):
run = PATTERNS[pattern]("keyword-2")
print(f"{pattern:12s} {len(run.calls):2d} calls {run.tokens:5d} tokens "
f"${run.cost:.5f} {run.wall_clock():.2f}s {run.correct}/6 right")
single 1 calls 555 tokens $0.00411 2.83s 1/6 right
sequential 6 calls 4190 tokens $0.01501 4.85s 4/6 right
supervisor 13 calls 3159 tokens $0.01464 4.16s 4/6 right
parallel 7 calls 2277 tokens $0.01170 3.75s 4/6 right
debate 25 calls 7061 tokens $0.03336 4.68s 4/6 right
Four of the five decompose the job, and all four get the same four sections right. Between the cheapest and the dearest of those four sits a factor of three in tokens.

What Just Happened?
The job and the corpus
Every section names the domain it belongs to and the question it answers. The briefing is structured input, which matters later.
Section(
name="regression_risk",
domain="risk",
brief="risk of the same failure recurring and what still has to change",
source="risk/change-register-0142",
)
source is the ground truth: the one document that carries the facts for that section. The corpus holds it alongside seventeen others, three of which were written to beat it on keyword overlap.
The scripted model and its limits
The stand-in model follows one rule.
def answer_section(section, context):
"""Right when the source document is in the context, wrong otherwise."""
if section.source in context:
return section.answer, True
return PLAUSIBLE_ERROR[section.name], False
That rule buys reproducibility: no API key, no sampling, byte-identical figures on every machine, and a token count that does not drift between runs. It costs realism in exactly one column. A model that can only fail by missing its source makes the accuracy figures a measurement of what each topology does to the context, and not a measurement of what a real model does with the context once it has it.
Read the accuracy column that way throughout. The cost and latency columns carry no such caveat, because call counts, token counts and the dependency graph are properties of the shape, and a live model changes none of them.
Every wrong answer is fluent and on topic, drawn from a decoy the retriever did put in the context. None is flagged as low confidence, which is the behaviour that makes this failure mode expensive in production.
The instrumentation
Token counting is a self-contained approximation of a byte-pair count, and latency comes from three published serving figures.
TTFT = 0.35 # time to first token
SECONDS_PER_OUTPUT_TOKEN = 0.012 # about 83 tokens per second
SECONDS_PER_PROMPT_TOKEN = 0.00008 # about 12,500 tokens per second
PRICE_IN, PRICE_OUT = 3.00, 15.00 # dollars per million tokens
Prices and speeds are for a mid-tier hosted model. Change them and every absolute number moves together, while the ratios between patterns stay where they are. The LLM cost calculator does the same arithmetic against current published rates, and the KV cache calculator covers the memory side of the same serving picture.
The scheduler is list scheduling over the dependency graph, with a concurrency cap.
def schedule(calls, concurrency=None):
"""Return the timeline and the makespan.
With no cap the makespan is the graph's critical path. With a cap it is
what an account with a rate limit gets.
"""
Going Deeper
The critical path, and what sits on it
The fan-out runs six workers at once and finishes in 3.75 seconds against the chain's 4.85. That is a speed-up of 1.30 times, from six-way parallelism. The missing factor is visible the moment the schedule is drawn.

The six workers write one or two sentences each and take between 0.68 and 0.94 seconds. The synthesis call reads all six sections and writes the whole briefing, 203 output tokens, and takes 2.81 seconds on its own. Parallelising the workers compresses the part of the job that was already short and leaves the long pole untouched.
This is Amdahl's law wearing agent clothing. The serial fraction of a fan-out is the reduce step, and the reduce step is where the writing happens. Any orchestration that ends by having one model write the deliverable inherits that floor, no matter how wide the fan-out above it.
The same arithmetic explains the supervisor's numbers. Its thirteen calls take 9.93 seconds of model time and 4.16 seconds of wall-clock, because the routers run together, the workers run together, and only the synthesis is alone. The debate panel compresses hardest of all, 21.41 seconds of model time into 4.68 seconds, which is the strongest argument in its favour.
The rate limit
Unlimited concurrency is an assumption, and it is the assumption that carries the fan-out's entire case. Cap the same graphs at two calls in flight and the ordering changes.

At two concurrent calls the fan-out takes 5.22 seconds and the chain still takes 4.85, so the pattern chosen for speed is now the slower of the two. The debate panel goes from 4.68 seconds to 12.27. The chain and the single agent do not move at all, because neither ever asked for concurrency.
Before adopting a fan-out for latency, check the rate limit it will run under and count the calls it will issue per job. A fan-out wider than the limit is a chain with extra bookkeeping.
The chain's context bill
The sequential pattern spends 4,190 tokens against the fan-out's 2,277 for identical output. The difference is not the call count, which is lower. Each call in a chain carries everything the chain has already retrieved and everything it has already written.

Cut the job into more sections and the chain's cost per section climbs by a factor of 2.42 between one section and six. The fan-out's falls to 0.87, because each worker sees its own section and nothing else, and the fixed system prompt is amortised over more work. The two lines cross between two sections and three. Past that the chain is paying for memory it mostly does not use.
Two things soften this in production. Prompt caching makes the chain's repeated prefix much cheaper to re-read, which is the single most effective fix and the reason the pattern survives at all. The mechanics are in KV cache memory and LLM serving. A chain whose later steps depend on earlier ones is also buying something with those tokens. The chain here does not, because the six sections are independent, which is the case where a fan-out belongs.
The router's two costs
The supervisor adds a routing call before every worker. Each one is cheap, 148 prompt tokens and a four-token answer, and together they are under a tenth of the bill. The real costs are elsewhere.
The first is structural: a router is a serial hop. No worker can start until its router finishes, which adds 0.41 seconds to the critical path and gets worse the moment routers and workers contend for the same rate limit.
The second is that the router is wrong sometimes, and a specialist cannot recover from it. The specialist cards here are the most frequent terms in each domain's own documents, which is how such descriptions get written in practice.
route(comms) # -> 'infra' (the section's documents live in 'customer')
Five of six sections route correctly. The comms section asks what customers were told about this incident, and the word "incident" belongs to the infrastructure specialist's vocabulary because infrastructure documents talk about incidents. The routed specialist owns only infrastructure documents, so the status note it needed was never reachable. No retrieval budget inside the wrong corpus recovers it.
That caps the supervisor at five of six sections on every retriever tried, including the best one, while every other decomposing pattern reaches six. A router does not add a failure mode so much as move the whole job's accuracy ceiling to whatever the router achieves.
The price of a panel
Debate is the most expensive pattern by a distance: 25 calls, 7,061 tokens, three times the fan-out's spend. On this workload it returns the same four sections.
The reason is mechanical. Three drafters given the same documents and the same deterministic model produce the same draft, so the majority is that draft, and the vote is a formality. Running one configuration three times and voting cannot beat running it once.
That much is a tautology in a deterministic testbed, so it does not settle the general question. The general question concerns drafters that differ by sampling instead, and it turns on how correlated their errors are. The chart below models it rather than measures it: each drafter's latent score is a shared component plus a private one, mixed by a correlation , with the marginal accuracy held fixed so only the correlation changes.

A drafter that is right 70% of the time, voted three ways, reaches 78.3% when the errors are independent. That is a real gain, and Condorcet's jury theorem promises it. At a correlation of 0.9 the same panel reaches 70.3%, a gain of three tenths of a point, for four times the calls. At a correlation of 1.0 it is worth nothing.
The shaded band is where three samples from one model at one prompt sit. Repeated samples share the training data, the tokeniser, the system prompt and the retrieved context, and their mistakes share all of those causes. A panel earns its cost through disagreement that comes from somewhere, so give the seats different retrievers, different tools or different models. Three temperature draws are three votes from one voter.
Retrieval against topology
All four decomposing patterns get four of six sections right on the naive retriever, and both failures are retrieval failures. The comms and regression_risk sections lose their source documents to decoys.
Run the whole grid, five retrievers against six orchestrations, and the picture separates cleanly.

Across the four topologies that decompose the job, switching topology moves mean accuracy by 0.4 sections. Switching retriever moves it by 1.75, more than four times as much. The vertical position of a point is set almost entirely by its colour.
Seven configurations reach six of six. The cheapest is one agent making one call against a hand-built section index, at $0.00597. The next cheapest is the fan-out on the same index at $0.01140, 1.9 times the price for the same result. The dearest is debate on the widest retriever at $0.05757, 9.6 times the price.
The section index is not an oracle. It names two candidate documents per section, and for four of the six one of the two is the wrong document. It took an afternoon with the wiki. Retrieving eight documents by keyword instead of two also reaches six of six, and costs nothing but prompt tokens. Either fix beats every orchestration in the table, and BM25 against embeddings is where the retrieval side of that gets its own treatment.
The uncomfortable reading is the one worth taking away. The supervisor pattern looks like specialisation, and its accuracy advantage on a naive retriever comes from routing giving each worker a domain-scoped index. That is a retrieval improvement wearing an orchestration costume, and the brief already declared the domain, so the router was recomputing a field its input contained.
Failure exposure
Cost and accuracy are the two columns people ask for. The third asks what a topology does with a wrong answer once it has one.

In a chain, a wrong section sits in the prompt of every call after it: 2.5 later calls on average, and five for the first section. In a fan-out or under a supervisor, one call sees it, the reduce step, and that call has no independent source to weigh it against.
The right panel is where debate earns something the cost table does not show. Its wrong draft reaches a judge that is holding two rivals for the same section, which is the only place in any of these five shapes where a wrong answer meets a competing one. Whether the judge picks correctly is the correlation question again. The structure at least makes the comparison possible.
Six rules, in order
The measurements collapse into an order of operations.
- Fix retrieval before choosing a topology. On this workload the retriever moved accuracy more than four times as far as the orchestration did, and the cheapest fix was one call with a better index. Orchestration that compensates for bad retrieval is expensive and it does not converge.
- Find the reduce step and time it. It is the serial floor under every fan-out. Six-way parallelism bought 1.3 times here because one synthesis call held 2.81 of the 3.75 seconds. Shorten the deliverable or split the write, or accept the floor.
- Count the calls against the rate limit. At two concurrent calls the fan-out became slower than the chain it replaced. A fan-out wider than the limit is a chain with extra bookkeeping.
- Use a chain only where later steps read earlier ones. Independent subtasks in a chain pay a context bill that rises with every step added, 2.42 times per section from one section to six. Prompt caching reduces the bill without changing the shape of the curve.
- Do not add a router for a field you already have. The router costs a serial hop before every worker and caps the job's accuracy at its own. Where the input already names the specialist, read it.
- Buy a panel only where the seats differ for a reason. Three samples of one model at one prompt sit near a correlation of one, where the majority is worth four tenths of a point at four times the cost. Different retrievers, tools or models make the vote mean something.
Where This Comes From
Anthropic's "Building Effective Agents" (December 2024) is the practitioner document this post is arguing with and mostly agreeing with. It separates workflows, where the steps are fixed in code, from agents, where a model chooses its own path, and its recommendation is to start with the simplest composition that works:
"When building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed."
Its named patterns map onto the four measured here: prompt chaining is the sequential chain, routing is the supervisor, parallelisation covers both the fan-out and the voting panel, and orchestrator-workers is the supervisor with a dynamic set of subtasks. The measurement above adds the arithmetic that the essay asserts.
Wu et al.'s "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation" (2023, arXiv:2308.08155) is the framework paper that made conversable agents a default. Its contribution is a programming model, agents that send each other messages with programmable reply behaviour, and the paper says plainly that the design space it opens is larger than the evidence for any point in it. Every topology in this post can be written in it, which is the point and also the problem, since the framework does not tell you which one to pick.
Du et al.'s "Improving Factuality and Reasoning in Language Models through Multiagent Debate" (2023, arXiv:2305.14325) is the strongest published case for the fourth pattern. Multiple instances propose and critique answers over several rounds, and the paper reports gains on arithmetic and factuality benchmarks. Read it next to the correlation chart above. The gains are real and they arrive through disagreement, so the operative question for a system is how much independent disagreement its panel can generate, and repeated samples of one model at one prompt generate little.
Hong et al.'s "MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework" (2023, arXiv:2308.00352) takes the specialisation idea furthest, encoding standard operating procedures and role definitions so that agents pass structured artefacts instead of free conversation. Structured handoffs are the part worth borrowing, and the section index in this post rests on the same idea.
The oldest result here belongs to none of them. Amdahl's law (1967) sets the speed-up available from parallelising a fraction of a job, and the synthesis call is the fraction that will not parallelise. Condorcet's jury theorem (1785) sets the accuracy available from a majority vote, and its independence assumption is the one that repeated sampling breaks.
Further reading
- The practitioner baseline: Anthropic, "Building Effective Agents", for the pattern catalogue and the argument for restraint
- The framework: Wu et al. (2023), sections 2 and 3 for the conversable-agent programming model
- The case for panels: Du et al. (2023), for what debate buys and on which benchmarks
- Structured handoffs: Hong et al. (2023), for encoding roles and artefacts rather than conversation
- Next topic to learn: evaluation for agent systems, since every number above is only as good as the ground truth behind it
Interactive Tools
- LLM Cost Calculator: Price your own orchestration against current published rates, since the factor of three between the cheapest and dearest pattern here is a token count you can reproduce
- KV Cache Calculator: Size the memory behind the context a sequential chain keeps re-reading, which is the other half of the chain's bill
- AI Readiness Assessment: Covers the governance questions a multi-agent design raises, including who approves what an agent does and what it is allowed to reach
Related Posts
- Building a ReAct Agent in Python: Reason, Act, Observe: The single agent this post composes, built from scratch in thirty-one lines
- Writing an MCP Server in Python Without the SDK: The protocol that lets specialists in a supervisor design share one tool surface
- Prompt Injection Against Tool-Using Agents: What happens when one of the documents a worker retrieves was written by an attacker
- BM25 vs Embeddings: Why Hybrid Retrieval Wins: The retrieval half of the result above, where the accuracy in this post mostly came from
- Mixture of Experts in LLMs: From Switch to DeepSeek-V3: Routing to specialists one layer down, inside the model instead of between agents
- LLM Serving: Continuous Batching & PagedAttention: Where the concurrency cap that reversed the fan-out's advantage comes from






