# AI human-review evaluation

Version 2.0 · Updated 2026-09-21 · Testing

Test whether the complete review workflow delivers the risk reduction being claimed.

Adapt this template to your intended use, applicable requirements, and approved procedures.

## When to use it

Use when you need to test whether the complete review workflow delivers the risk reduction being claimed.

## Information needed before starting

- Defined AI use, authority limits and assessed configuration
- Controlled sources and independent reference results relevant to this worksheet
- Applicable risk assessment, acceptance basis and responsible reviewers

## Field instructions

- **Task and proposed reliance:** Define what reviewers must decide and what AI output may influence.
- **Reviewer prerequisites:** Record process knowledge, source access, AI limitations, time, authority and procedural training.
- **Cases and expected results:** Identify representative cases, independently adjudicated reference results and scoring rubric.
- **Design and independence:** Record users, conditions, assignments, order and measures against answer leakage.
- **Acceptance criteria:** Set outcome and critical-failure rules before testing; justify them for the intended use.
- **Observed outcome:** Score final correct/false/missed findings, source verification and escalation.
- **Limits and deviations:** Retain dependency/coverage limits, failures and open corrective actions.
- **Decision and retest:** State reliance restrictions and the fresh evidence needed after changes.

## Completed fictional example

Synthetic HR-01 uses six reviewers, each completing two assisted and two unassisted sessions. Twelve documents appear once per condition with different reviewers: 24 sessions total. Each condition covers 40 reference issues including ten critical.

| Measure | Assisted final review | Unassisted final review |
| --- | --- | --- |
| Correct findings | 36 | 37 |
| Missed issues | 4 | 3 |
| Critical issues detected / reference | 9 / 10 | 10 / 10 |
| Critical misses | 1 | 0 |
| False findings remaining | 2 | 1 |
| Decision | Do not approve expanded reliance; DEV-02/03 remain open | Small study does not prove zero risk or causal superiority |

## Related guidance

- [Read chapter 15: Human Review and Oversight](https://csvtocsa.com/ai-assurance-academy/ai-in-csv-csa-chapter-15-human-oversight-and-automation-bias)

- [Read chapter 20: AI Release Decision: Complete Case Study](https://csvtocsa.com/ai-assurance-academy/ai-in-csv-csa-chapter-20-end-to-end-case-study-and-inspection-readiness)

Canonical page: https://csvtocsa.com/worksheets/ai-human-review-evaluation