Diagnosing Rag Failure Modes

Diagnosing RAG Failure Modes is a skill that helps engineers and AI architects determine why a RAG pipeline is returning poor results — and whether the root cause is implementation quality or a fundamental architectural mismatch. It provides a two-class query taxonomy that splits queries into Class A (factual lookup, RAG-safe) and Class B (relational or temporal, RAG danger zone), giving teams a shared vocabulary to route queries appropriately before tuning begins. A five-question diagnostic checklist — covering whether a multi-document join is required, whether order matters, whether a causation chain is involved, whether a time span is relevant, and whether the query asks why rather than what — scores any failing query as Class A, borderline, or Class B. The skill defines four named failure patterns: multi-hop relational failure, temporal sequencing failure, organizational context failure, and scale failure. Each pattern comes with a symptom description, a worked example, and a specific architectural fix. Results are captured in a Failure Classification Report template that structures the query, its class, matched failure patterns, a root cause paragraph, and a recommended fix — making the diagnosis communicable to both engineers and non-technical stakeholders. Each failure pattern also references a companion skill for remediation, so diagnosis connects directly to next steps. This skill is most relevant when a RAG pipeline returns plausible-sounding but incorrect results on relational or temporal queries, or when a team is iterating on embeddings and chunk size without improvement.