Framework: AI Workflow Failure Mode Audit
A practical audit framework for diagnosing where AI workflows fail across context, tools, review, source quality, and operational design.
Use cases
Operations & Workflow, Strategy & Planning
Platforms
Model-Agnostic, Tool-Agnostic
Jump to a section
The resource
Copy and adapt. Do not paste blind.
Audit every failing workflow in this order:
1. Was the task defined clearly?
2. Was the right context available at the right step?
3. Were sources reliable and current?
4. Did the workflow use the right tool or model for the job?
5. Was there a review or approval gate where one was needed?
6. Did the handoff between steps preserve the necessary information?
7. Was the failure actually upstream process design dressed up as an AI problem?
Return:
- failure point
- likely cause
- confidence
- cheapest fix
- test to validate the fixWhen to Use This
Use this when an AI workflow is underperforming and the team keeps blaming “the model” without a disciplined way to inspect the real fault line.
It is useful for prompt chains, agents, automations, support flows, and content systems that sometimes work and sometimes fall apart.
Why It Works
The framework starts with structure and context before it looks at models. That is deliberate. Most failures are architectural before they are vendor-specific.
The “cheapest fix” requirement keeps the audit practical. Diagnosis without action is just theatre.
How to Customise
Add category-specific checks if you work in support, legal, compliance, or customer-facing automation.
If you already track incidents, link the audit to a standard post-mortem template so fixes can be compared over time.
Limitations
This framework improves diagnosis, but it does not guarantee the team will choose the right fix.
It also depends on having real evidence from the failing workflow, not just vague complaints.
Model Notes
Model-agnostic and tool-agnostic. The value is in the audit sequence rather than the platform.
Related Resources
Browse FrameworksFramework: Prompt Audit Checklist
A 15-point checklist for evaluating any prompt before putting it into production. Catches the most common prompt failures: vague instructions, missing constraints, absent error handling, and untested edge cases.
Operations & Workflow · Strategy & Planning
Framework: Context Engineering Checklist
A checklist for deciding what context a model actually needs, how to structure it, and what should be left out.
Development & Code · Strategy & Planning
Framework: AI Implementation Planning Canvas
A planning canvas for choosing the right workflow, ownership, data inputs, risks, and success metrics before building.
Strategy & Planning · Operations & Workflow