# AI performance evaluation

Version 2.0 · Updated 2026-09-21 · Testing

Record issue-level results and retain a traceable basis for metrics and the decision.

Adapt this template to your intended use, applicable requirements, and approved procedures.

## When to use it

Use when you need to record issue-level results and retain a traceable basis for metrics and the decision.

## Information needed before starting

- Defined AI use, authority limits and assessed configuration
- Controlled sources and independent reference results relevant to this worksheet
- Applicable risk assessment, acceptance basis and responsible reviewers

## Field instructions

- **Requirement or risk reference:** Link the requirement and failure being evaluated.
- **Test category:** State normal, negative, boundary, source, repeatability or other defined category.
- **Reference result:** Record the independently established expected finding and its source.
- **AI result:** Record the actual finding or omission and execution/configuration identifiers.
- **Severity:** Use the predefined local rubric and keep critical results separate.
- **Outcome:** Record correct, false, missed or unresolved, with adjudication rationale.
- **Deviation reference:** Link failed criteria and required actions; do not hide failures from totals.
- **Conclusion:** Explain permitted reliance, exclusions and unresolved uncertainty.
- **Units and design:** Record distinct documents, reference issues and repeated runs separately; identify dependencies.

## Completed fictional example

Synthetic T-01 includes 12 documents and 40 reference issues: 10 critical and 30 major. The AI reports 38 findings: 32 correct and six false. Eight issues are missed, including two critical. Precision 84.2%; recall 80%; critical recall 80%. The rows below are illustrative extracts, not the entire 40-issue register.

| Requirement / risk | Test category | Reference result | AI result | Severity | Outcome | Deviation | Conclusion |
| --- | --- | --- | --- | --- | --- | --- | --- |
| REQ-01 / R-01 | Critical omission | Protocol must challenge specified alarm condition | No candidate finding | Critical | Missed | DEV-01 | Expanded reliance unsupported |
| REQ-02 / R-02 | Unsupported statement | No approved source requires screenshot for every test | Claims every test needs screenshot | Major | False | DEV-03 | Reject unsupported claim |
| REQ-01 / R-01 | Requirement mismatch | Acceptance criterion differs from approved requirement | Identifies mismatch with correct source | Major | Correct | None | Evidence for this case only |

## Related guidance

- [Read chapter 9: Reference Data and Expected Results](https://csvtocsa.com/ai-assurance-academy/ai-in-csv-csa-chapter-9-supplier-and-foundation-model-assurance)

- [Read chapter 11: Measuring AI Performance](https://csvtocsa.com/ai-assurance-academy/ai-in-csv-csa-chapter-11-performance-evaluation-and-metric-selection)

- [Read chapter 12: AI Test Strategy](https://csvtocsa.com/ai-assurance-academy/ai-in-csv-csa-chapter-12-verification-strategy-challenge-sets-and-repeatability)

Canonical page: https://csvtocsa.com/worksheets/ai-performance-evaluation