AI performance evaluation
Record issue-level results and retain a traceable basis for metrics and the decision.
Adapt this worksheet to your applicable procedures, intended use and requirements. Fictional examples are teaching material, not execution evidence.
Field instructions
- Requirement or risk reference: Link the requirement and failure being evaluated.
- Test category: State normal, negative, boundary, source, repeatability or other defined category.
- Reference result: Record the independently established expected finding and its source.
- AI result: Record the actual finding or omission and execution/configuration identifiers.
- Severity: Use the predefined local rubric and keep critical results separate.
- Outcome: Record correct, false, missed or unresolved, with adjudication rationale.
- Deviation reference: Link failed criteria and required actions; do not hide failures from totals.
- Conclusion: Explain permitted reliance, exclusions and unresolved uncertainty.
- Units and design: Record distinct documents, reference issues and repeated runs separately; identify dependencies.
Completed fictional example
Synthetic T-01 includes 12 documents and 40 reference issues: 10 critical and 30 major. The AI reports 38 findings: 32 correct and six false. Eight issues are missed, including two critical. Precision 84.2%; recall 80%; critical recall 80%. The rows below are illustrative extracts, not the entire 40-issue register.
| Requirement / risk | Test category | Reference result | AI result | Severity | Outcome | Deviation | Conclusion |
|---|---|---|---|---|---|---|---|
| REQ-01 / R-01 | Critical omission | Protocol must challenge specified alarm condition | No candidate finding | Critical | Missed | DEV-01 | Expanded reliance unsupported |
| REQ-02 / R-02 | Unsupported statement | No approved source requires screenshot for every test | Claims every test needs screenshot | Major | False | DEV-03 | Reject unsupported claim |
| REQ-01 / R-01 | Requirement mismatch | Acceptance criterion differs from approved requirement | Identifies mismatch with correct source | Major | Correct | None | Evidence for this case only |
Blank worksheet
System / record ID: ____________________
Prepared by / date: ____________________
Required review / approval under local procedure: ____________________
| Field | Your entry |
|---|---|
| Requirement or risk reference | ____________________ |
| Test category | ____________________ |
| Reference result | ____________________ |
| AI result | ____________________ |
| Severity | ____________________ |
| Outcome | ____________________ |
| Deviation reference | ____________________ |
| Conclusion | ____________________ |
| Units and design | ____________________ |
Blank results or assessment rows
| Requirement / risk | Test category | Reference result | AI result | Severity | Outcome | Deviation | Conclusion |
|---|---|---|---|---|---|---|---|
| ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ |
| ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ |
| ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ | ____________________ |