Why Multi-Agent Systems Break at the Handoff

You shipped a multi-agent system. Each agent passes its unit tests in isolation — the researcher finds good sources, the analyst does sound math, the writer produces clean prose. Then you run the full pipeline and the final output is subtly wrong. Not obviously broken, not throwing errors, just… off. Every span in your trace shows status: ok. Nobody can tell you which agent is at fault, because none of them did anything wrong on their own.

This is the debugging experience that’s become routine for teams running orchestrator/worker, parallel-with-merge, or peer-handoff agent architectures in production. And it’s not a symptom of immature models or bad prompts. A research taxonomy built from 1,600+ annotated traces across seven popular multi-agent frameworks — AutoGen, CrewAI, LangGraph, OpenAI Swarm, and others — found that roughly 79% of multi-agent failures trace back to specification and coordination problems, not model capability or infrastructure faults. The bug lives in the space between agents, at the handoff, where nobody’s logs cover the boundary.

That finding should reshape how you think about agent architecture reviews. If four out of five production failures happen at the seams rather than inside any component, then code review, prompt tuning, and model upgrades are solving the wrong 20% of the problem. The seams need their own engineering discipline.

The taxonomy: what actually breaks

The Multi-Agent System Failure Taxonomy (MAST), developed by researchers at UC Berkeley and validated with high inter-annotator agreement (kappa = 0.88) across 150 hand-labeled traces before scaling to 1,600+ with an LLM-as-judge pipeline, groups failures into three categories: system design issues, inter-agent misalignment, and task verification. Fourteen distinct failure modes sit under those three buckets, and the inter-agent misalignment category — communication breakdowns, context loss during handoffs, conflicting outputs, format mismatches between agents — accounts for the largest single share of observed failures.

The pattern shows up cleanly in production debugging writeups too. One widely circulated case study, published by the observability vendor Sentry, walks through a three-agent research system (an Advocate, a Skeptic, and a Synthesizer that reads both blind and produces a balanced analysis) that kept producing lopsided output. Every individual agent’s logs looked fine: reasonable latency, reasonable token counts, no errors. The actual cause was that the Skeptic’s web_search tool call returned a single weak result on one query, while the Advocate’s returned three or four strong citations across multiple queries. The Synthesizer, given one well-sourced brief and one thin one, did exactly what a competent reader would do: it weighted the better-sourced argument more heavily. The bug was in a tool call inside one agent that silently degraded the input to a different agent two steps downstream. Nothing threw an exception. The failure was invisible unless you compared raw tool outputs across the boundary.

That’s the shape of the problem: not agents reasoning badly, but agents reasoning correctly over corrupted, incomplete, or asymmetric inputs that arrived through a handoff nobody was watching.

Why single-agent monitoring can’t see this

Standard agent observability tells you an agent ran for 3.1 seconds and consumed 2,400 tokens. It doesn’t tell you that the agent’s tool call returned degraded data, that its output was thinner than a peer agent’s, or that a downstream agent will silently propagate that weakness into a confidently wrong final answer. Three structural reasons make this hard to catch with the tooling most teams already have.

Blame is distributed across the graph. When output is wrong, no single agent’s logs show the defect, because each agent did reasonable work given what it received. The defect exists only in the relationship between two spans, and most tracing tools are built to show you spans, not relationships.

The worst failures produce plausible output. In traditional software, broken states throw errors. In multi-agent systems, a degraded intermediate result looks like a normal intermediate result — a thin research brief instead of an error, a slightly-off classification instead of a crash — and it gets incorporated without friction into everything downstream.

The state space is combinatorially large. A single agent with five tools has five possible actions per step. Three agents with five tools each, running in parallel with merged results, produce far more execution paths than you can enumerate in a test suite. You can’t pre-test every combination; you can only observe what happens in production and instrument for it.

Architecture Impact

What changes in system design? Agent boundaries need to be treated as first-class architectural surfaces, not implementation details of an orchestration framework. That means explicit contracts for what crosses a handoff (structured data vs. free-text summaries), instrumentation at every invoke_agent and handoff span rather than only at the top-level request, and a decision, made deliberately, about whether a given task actually needs multiple agents or is a single agent with extra prompt complexity in disguise.

What new failure mode appears? Silent cascading degradation: an upstream agent returns a plausible-but-thin result, a downstream agent accepts it without question, and the final output is confidently wrong while every span reports success. Context drift is the sibling failure mode — each summarization step between agents is lossy, and after two or three handoffs the original user intent can be barely recognizable, with no single log entry showing where it went missing.

What enterprise teams should evaluate:

• Platform/MLOps teams: whether their tracing captures full prompts and tool outputs at every agent boundary, not just token counts and latency, and whether they can diff inputs across parallel agents that are supposed to be doing comparable work.
• Architecture/engineering leads: whether each multi-agent split is justified by the five conditions that actually warrant separation — conflicting objectives, required information isolation, different models per role, genuine parallelism, or hard security boundaries — versus a single agent with more tools.
• Risk/compliance teams in regulated workflows (credit decisioning, claims processing, trade documentation): whether an audit trail exists that reconstructs what data crossed each handoff, since “the agent said so” won’t satisfy a regulator asking why a multi-step automated decision went wrong.

Cost / latency / governance / reliability implications: Multi-agent systems carry superlinear coordination overhead: a Synthesizer-style aggregation step running a larger model can account for 50–60% of total pipeline cost despite executing once per request, while the worker agents feeding it run cheaper models many times over — invisible without per-agent cost attribution. On governance, Gartner’s now widely cited estimate that the average Fortune 500 enterprise will run more than 150,000 agents by 2028, against roughly 13% of organizations reporting they have adequate agent governance today, is the numeric expression of exactly this boundary problem at scale: nobody can govern a handoff they can’t see.

Common Failure Modes

Three architectural patterns account for most production multi-agent deployments, and each fails differently.

Orchestrator/worker, where one agent routes subtasks to specialists, breaks when the orchestrator misclassifies a task and routes it to the wrong specialist — who then does competent work on the wrong problem — or when it passes insufficient context and the specialist fills the gap with a hallucinated assumption.

Parallel-with-merge, where independent agents work concurrently and a final agent combines their output, breaks on uneven tool quality between the parallel branches. The merge agent has no way to know its inputs were unequal; it only sees finished text, not the raw data quality underneath. This is the exact failure from the Sentry case study above, and it is close to undetectable without instrumentation that lets you compare sibling agents’ tool-call counts and result richness side by side.

Peer handoffs, where agents transfer control directly to one another (the pattern behind frameworks’ native handoff() primitives), break at the state-transfer decision itself: full conversation history blows the context window, a summary loses nuance, and a bug in what gets carried across the handoff looks, to anyone debugging it, exactly like a bug in the receiving agent’s reasoning.

Implementation Guide

Start by instrumenting agent boundaries before you instrument anything else. If you’re on OpenAI’s Agents SDK, LangGraph, CrewAI, or a comparable framework, adopt the OpenTelemetry gen_ai semantic conventions (gen_ai.invoke_agent, gen_ai.execute_tool, gen_ai.handoff) and turn on full prompt-and-response capture, not just token and latency metrics — vendors like Sentry, Langfuse, and LangSmith all support this natively now. Token counts tell you cost; the actual prompt and tool-output text is where the boundary defects live. This is the single highest-leverage change you can make, because it’s the difference between debugging by re-reading code and debugging by comparing what actually crossed each seam.

The mistake to avoid is treating multi-agent architecture as a default rather than a decision. The MAST research and the accompanying industry guidance (Microsoft’s own Cloud Adoption Framework makes this point explicitly) both land on the same conclusion: distinct roles suggest multiple agents, they don’t automatically justify a multi-agent architecture. Multi-agent splits earn their coordination overhead only when objectives genuinely conflict (a generator and a critic sharing a prompt produces mediocre output at both jobs), when information must be isolated (blind review requires separate context windows), when different models legitimately serve different roles, when tasks are genuinely parallelizable, or when security boundaries require it. If none of those apply, a single agent with more tools and a better system prompt will out-perform a multi-agent pipeline on cost and latency, and it removes the entire class of boundary failures described here.

You’ll know the approach is working when you can answer, for any bad output, a specific question in under five minutes from traces alone: which agent received degraded or asymmetric input, and what did that input look like compared to its peers. If that question still requires re-running the pipeline with print statements, your observability isn’t at the boundary yet. A practical proxy metric: build a per-agent tool-failure-rate dashboard grouped by agent name, not globally. A tool with a 5% global failure rate that spikes to 20% for one specific agent — because of how that agent formulates its queries — is invisible in an aggregate number and obvious in a segmented one.

Over a six-to-twelve month maturity path, teams that get this right converge on a small set of practices: structured data contracts between agents instead of natural-language summaries wherever context must survive a handoff; per-agent cost and latency dashboards that make coordination overhead visible rather than folded into a single pipeline total; sampling AI traces near 100% rather than the 10-20% typical of conventional application tracing, because a failure mode that appears in 1 of 50 runs needs hundreds of samples to catch at low sampling rates; and an explicit architectural review gate that requires justifying every additional agent against the five conditions above rather than adding agents by default. Enterprise governance vendors are already building for this world — WSO2’s Agent Manager, which reached general availability on September 15, 2026, ships MCP-level governance and per-agent identity specifically because stitched-together gateway-plus-observability-plus-identity tooling doesn’t cover the full agent lifecycle or the boundaries between agents. Whether you buy a control plane or build the instrumentation yourself, the target is the same: make the handoff observable before you make it smarter.

Sources

• Why Do Multi-Agent LLM Systems Fail? — arXiv:2503.13657, MAST taxonomy and MAST-Data (1,600+ annotated traces, 7 frameworks)
• Debugging multi-agent AI: When the failure is in the space between agents — Sentry Blog
• WSO2 Agent Manager Brings Sovereign AI Governance to Enterprise Agent Sprawl — GlobeNewswire, September 15, 2026