Complex assurance scenario
RAG validation-document reviewer
Assuring source-grounded findings without delegating the compliance decision.
Context
A retrieval-augmented assistant reviews validation documents for missing approvals, inconsistencies, and traceability gaps.
Intended Use
Prioritize potential review findings for disposition by a qualified human; it cannot approve documents.
Boundary
Document parsing, chunking, index, retrieval, model, prompt, citation rendering, user interface, export, access, and human workflow.
Failure Chain
Relevant approval page is misparsed → retrieval omits it → model reports a missing approval → reviewer trusts the finding → unnecessary remediation is initiated.
Existing Controls
- Read-only source documents
- source citations
- human disposition
- role access
- approved model and prompt versions
Evidence
- Representative labeled challenge set
- citation-grounding tests
- document-format stress tests
- false-negative review
- human-factors evaluation
Evidence Gaps
Performance for scanned tables with handwritten initials is uncertain.
Test Design
Stratify by format and document type; seed known gaps; challenge prompt injection; measure finding recall, false findings, citation correctness, and reviewer override.
Decision
Restrict scanned handwritten records to manual review until OCR evidence is adequate.
Residual Risk
A plausible, cited interpretation may still be professionally wrong.
Monitoring
Overrides, confirmed misses, citation failures, document-format drift, latency, and model changes.
Reassessment Triggers
- Model, prompt, parser, embedding, or corpus change
- New document type
- Miss threshold breach
Inspection Questions
- How are false negatives estimated?
- Can every finding be traced?
- Which decisions are prohibited?