FDE-008field-patterns/customer-facing-ai-acceptance-test-plan.mdUPDATED: 06/18/2026
Customer-Facing AI Acceptance Test Plan
Pattern
Name: Customer-facing AI acceptance test plan
When to use it: When an AI workflow needs customer approval before rollout or broader adoption.
Why it matters for FDE roles: Acceptance criteria turn AI quality from a subjective reaction into a shared delivery target.
Plain-English Description
An AI acceptance test plan defines the examples, review criteria, expected behavior, failure categories, and signoff process for an AI workflow.
Situation Signals
- Job listing signal: evals, testing, AI reliability, quality measurement.
- Customer signal: the customer asks whether the AI is accurate or safe enough.
- Project signal: demo quality is promising but launch criteria are unclear.
What To Ask
- What tasks should the AI be judged on?
- What examples represent normal, hard, and risky cases?
- What makes an answer acceptable, editable, or rejected?
- Who has authority to sign off?
What To Do
- Build a small evaluation set with customer examples.
- Define review categories and scoring criteria.
- Include source-grounding and uncertainty checks.
- Track accepted, edited, rejected, and escalated outputs.
Artifacts To Produce
- Diagram: evaluation and approval flow.
- Checklist: acceptance criteria and failure categories.
- Demo/prototype: review queue with sample outputs.
- Customer-facing note: test plan and signoff summary.
Failure Modes
- Testing only easy examples.
- Measuring correctness without measuring usefulness.
- No subject-matter expert review.
- Launch criteria are decided after the demo.
Interview Language
One sentence I could say in an interview:
I like customer-facing AI evals that use real examples, explicit acceptance criteria, source checks, and a clear approve/edit/reject workflow.
Relevant work experience for this pattern: