Compound AI Failures Live at the Seams, Not in the Model

Open the dashboard for your production RAG or agent system and look at what is green. Model endpoint latency, healthy. Vector store, healthy. Tool API error rate, healthy. Every component passes its own health check, and yet a user is staring at a confidently wrong answer. This is the most common shape of an AI incident, and it is the one our monitoring is least built to see: every part works, and the system does not.

The reason is structural. A compound AI system is a chain of components, a retriever, a ranker, a model, a tool layer, an orchestrator, and each hands the next one an artifact that is only loosely typed and almost never validated. Text goes in, text comes out, and a 200 OK is returned either way. The component that broke is not the component that looks broken, and the component that looks broken is usually just the one holding the bad input.

A recent paper makes this measurable. Paul and Nandy analyzed 150 production incident reports from open-source compound AI projects and anonymized enterprise deployments (arXiv 2610.02503, submitted October 1, 2026). They built a taxonomy of 23 failure modes in five categories and then tested resilience patterns against each with fault injection. The headline conclusion matches what most of us have learned the expensive way: failures originate at component boundaries, not inside individual components. The corollary is the useful part. If the seams are where systems fail, the seams are where reliability engineering has to live.

Where the 150 Incidents Actually Landed

The distribution is flatter than most teams assume. Retrieval accounted for 38 incidents (25%), generation for 31 (21%), tools for 29 (19%), orchestration for 28 (19%), and integration for 24 (16%). If you have been treating reliability as primarily a model-quality problem, note that no single category dominates, and the largest one is upstream of the model.

The top individual modes are instructive. Index drift and stale embeddings led at 10.7% of all incidents. Hallucination under noisy context was 8.0%, API timeout cascades 6.7%, infinite retry or loop deadlock 6.0%, and schema drift and type coercion errors each 5.3%. Almost none of these are “the model got worse.” They are what happens when a job changes an upstream artifact and nothing downstream notices.

One enterprise incident in the study captures the pattern. A re-indexing job silently switched embedding models. Mean query-to-document cosine similarity fell from 0.78 to 0.41, and generation errors went undetected for nine days. No component threw an error. The retriever returned results, the model generated fluent answers from them, and the answers were wrong. The paper also reports that retrieval failures carry a 2.3x multiplier on downstream generation errors, which is why the first seam in a RAG pipeline deserves the most protection.

Silent Degradation Is the Dominant Failure Mode

Three cross-cutting patterns matter more than the category counts. Silent degradation appeared in 51% of incidents. Cascade amplification, where one failure spreads across components, appeared in 34%. Semantic errors that pass validation appeared in 22%.

The detection numbers are the ones to put in front of your leadership. Crash failures were detected in a mean of 12 minutes. Silent degradation took a mean of 4.2 days. That is roughly a 500x gap in time-to-detection, and it means your incident response process, your on-call rotation, and your SLO math were all designed for the failure class that is the minority.

The cascade cases are equally uncomfortable. In 8 of 10 tool-timeout incidents, a single slow tool cut total system throughput by 60 to 80 percent within minutes, because the orchestrator held workers waiting on it and starved everything else. A retry loop with no exponential backoff, triggered by a malformed tool response that was missing from the retry exclusion list, cost one team $2,400 in API fees over a weekend. And a float round-trip that turned a 0.85 threshold into 0.8500000000000001 silently dropped 23% of results. None of these are exotic. They are what boundaries do when nobody owns them.

Resilience Patterns and What They Measured

The authors injected faults into a six-component testbed, 100 trials per scenario, and measured five patterns. Circuit breakers reduced cascade depth by 89%, from a mean of 3.8 components affected to 0.4. Output quality gates caught 73% of silent degradation before user impact. Component isolation cut blast radius by 64%, dropping the share of concurrent requests affected from 78% to 28%. Semantic validators caught 81% of semantic errors that had passed schema validation. Typed interfaces reduced integration failures by 92%.

Two findings should shape how you sequence the work. First, “no single pattern addresses more than 40% of failure modes alone,” so this is a layered investment, not a tool purchase. Second, systems with three or more patterns in place had a mean time to recovery of 8.4 minutes versus 28.7 minutes for unstructured monitoring, a 71% reduction. The caveat is honest and worth repeating: the baseline was unstructured monitoring rather than standard retry-with-backoff, the fault injection ran on one testbed rather than live production, and the enterprise sample was only 53 incidents from a small number of organizations. Treat the percentages as directional, not as numbers to put in a business case.

The Cost of Putting Gates at the Seams

None of this is free, and the paper quantifies the overhead, which is rare. Quality gates add a median 120 ms per request. Semantic validators add 80 to 150 ms. Typed interfaces add 5 to 15 ms per boundary crossing. Component isolation raises peak memory by about 18%. Circuit breakers add under 2 ms but impose a 30-second cool-down when tripped.

For a voice agent or a latency-sensitive autocomplete path, 120 ms plus 150 ms of validators is a real budget line. The paper’s guidance is the right one: run quality gates asynchronously for alerting on latency-sensitive paths, and reserve synchronous gates for high-stakes outputs where a wrong answer costs more than a quarter of a second. That is a per-route decision, not a global one.

Architecture Impact

What changes in system design? Every inter-component hop becomes a contract with a type, a validator, and a policy for what happens when the contract is violated. The retriever-to-generator edge gets a relevance gate (the paper used a mean cosine similarity threshold of 0.62), the tool layer gets per-tool circuit breakers and isolated connection and rate-limit pools, and every payload crossing a boundary is parsed into a typed schema rather than passed as a string or dict. Reliability logic moves out of the individual components and into the glue between them.

What new failure mode appears? Gate drift. A static similarity threshold tuned for one embedding model will quietly misfire after a re-index or a model swap, either blocking good traffic or passing bad. Circuit breakers that trip on HTTP status alone will stay closed through the most common AI failure, a 200 OK with degraded content. The gates themselves need versioning and monitoring, or they become another silent component.

What enterprise teams should evaluate:

• Platform and SRE teams: whether circuit breakers key off semantic quality signals (retrieval relevance, output coherence) and not only status codes and latency, and whether resource pools are isolated per tool.
• Data and ML engineering teams: whether re-indexing, embedding model changes, and schema changes run behind a similarity-distribution check before they go live.
• Application and agent developers: whether every boundary has a typed, runtime-validated interface, and whether retry policies have exponential backoff and an explicit exclusion list for malformed responses.

Cost / latency / governance / reliability implications: Expect 120 ms median for synchronous quality gates and 80 to 150 ms for semantic validators, so budget them only on routes where the cost of a wrong answer justifies it. Component isolation costs roughly 18% more peak memory. On the reliability side, the study’s 71% MTTR reduction (28.7 to 8.4 minutes) for teams with three or more patterns, and the gap between 4.2 days and 12 minutes to detect silent versus loud failures, is the case for the spend. Governance benefits too: a gate that logs why it blocked or passed an output is an audit trail you did not have before.

Common Failure Modes

The implementations tend to go wrong in predictable ways. Teams add a quality gate with a threshold copied from a blog post and never recalibrate it, so it passes everything after the next embedding upgrade. They place a circuit breaker around the model endpoint, where the provider already provides stability, and leave the flaky internal tool unprotected. They validate schemas but not semantics, which means the 22% of incidents where a well-formed but wrong value flows through remain invisible. And they isolate compute but share a rate-limit budget, so one noisy tool still starves the rest during a retry storm.

Implementation Guide

Start with the seam that carries the most risk and the least visibility, which in most systems is retriever to generator. Log the similarity score distribution of every retrieval, not just the top-k results, and alert on movement in the distribution rather than on an absolute value. A drop in mean cosine similarity from 0.78 to 0.41 is unmistakable on a daily histogram and invisible everywhere else. Once you have a week of baseline, add a gate that routes low-relevance retrievals to a fallback, such as a broader search, a refusal, or a human, instead of letting the model improvise over weak context.

Next, put typed interfaces on every boundary, because they are the cheapest pattern and the paper measured the largest effect, a 92% reduction in integration failures for 5 to 15 ms per crossing. Use Pydantic or an equivalent to parse tool responses, retrieval payloads, and inter-agent messages. Reject on mismatch and name the failing boundary in the error. The float round-trip that dropped 23% of results is exactly what this prevents, and it is cheap to adopt incrementally, one edge at a time.

Then handle tool failure. Wrap each tool in its own circuit breaker with its own timeout, concurrency limit, and rate-limit budget, so a slow dependency cannot take the orchestrator’s worker pool with it. Make the breaker semantic where you can: treat empty results, schema violations, and sustained low-quality outputs as failures, not just 5xx responses. Add exponential backoff and a retry exclusion list for response shapes that will never succeed on retry. What to avoid is the premature global solution: do not put a synchronous LLM-as-judge gate on every response before you know which routes are high-stakes, because you will spend the latency budget on traffic that did not need it.

You will know it is working when three signals move. Time-to-detection for quality incidents should fall from days toward hours, because gates and distribution alerts surface drift that user complaints used to. The blast radius of a single tool failure should shrink, and you can verify that directly by injecting a timeout into one tool in staging and watching whether throughput on unrelated routes holds. And your incident postmortems should start naming a boundary and a contract rather than “the model hallucinated.”

Over six to twelve months, teams that get this right end up treating fault injection as a standing practice. They keep a catalog of their own failure modes mapped to the five categories, run scheduled chaos tests against the retriever, tool, and orchestrator seams, and version gate thresholds alongside embedding models and prompts. The mature end state is a system where a re-index cannot reach production without passing a distribution check, where every boundary has an owner, and where silent failure is the exception that gets a postmortem rather than the norm that gets discovered by a customer.

Sources

• Compound AI System Reliability: A Failure Taxonomy and Resilience Pattern Catalog from 150 Production Incidents (arXiv 2610.02503)
• Full text HTML, arXiv 2610.02503