AI human-review evaluation
Test whether the complete review workflow delivers the risk reduction being claimed.
Adapt this worksheet to your applicable procedures, intended use and requirements. Fictional examples are teaching material, not execution evidence.
Field instructions
- Task and proposed reliance: Define what reviewers must decide and what AI output may influence.
- Reviewer prerequisites: Record process knowledge, source access, AI limitations, time, authority and procedural training.
- Cases and expected results: Identify representative cases, independently adjudicated reference results and scoring rubric.
- Design and independence: Record users, conditions, assignments, order and measures against answer leakage.
- Acceptance criteria: Set outcome and critical-failure rules before testing; justify them for the intended use.
- Observed outcome: Score final correct/false/missed findings, source verification and escalation.
- Limits and deviations: Retain dependency/coverage limits, failures and open corrective actions.
- Decision and retest: State reliance restrictions and the fresh evidence needed after changes.
Completed fictional example
Synthetic HR-01 uses six reviewers, each completing two assisted and two unassisted sessions. Twelve documents appear once per condition with different reviewers: 24 sessions total. Each condition covers 40 reference issues including ten critical.
| Measure | Assisted final review | Unassisted final review |
|---|---|---|
| Correct findings | 36 | 37 |
| Missed issues | 4 | 3 |
| Critical issues detected / reference | 9 / 10 | 10 / 10 |
| Critical misses | 1 | 0 |
| False findings remaining | 2 | 1 |
| Decision | Do not approve expanded reliance; DEV-02/03 remain open | Small study does not prove zero risk or causal superiority |
Blank worksheet
System / record ID: ____________________
Prepared by / date: ____________________
Required review / approval under local procedure: ____________________
| Field | Your entry |
|---|---|
| Task and proposed reliance | ____________________ |
| Reviewer prerequisites | ____________________ |
| Cases and expected results | ____________________ |
| Design and independence | ____________________ |
| Acceptance criteria | ____________________ |
| Observed outcome | ____________________ |
| Limits and deviations | ____________________ |
| Decision and retest | ____________________ |
Blank results or assessment rows
| Measure | Assisted final review | Unassisted final review |
|---|---|---|
| ____________________ | ____________________ | ____________________ |
| ____________________ | ____________________ | ____________________ |
| ____________________ | ____________________ | ____________________ |