Version 2.0 · Updated 2026-09-21 · Testing
Record expected results, AI findings, missed issues, and performance measures for a defined task.
Adapt this template to your intended use, applicable requirements, and approved procedures.
When to use it
Use before documenting the corresponding assessment or decision.
- Defined task, assessment unit and supported document formats
- Independent reference issues with severity and predefined matching rules
- Recorded AI findings, missed issues and complete workflow observations
Field instructions
- Task: Describe the specific task, users, outputs and decisions supported. Identify the assessed environment and limits of reliance.
- Assessment unit: Define the unit being scored, such as an individual issue, document or final review decision. Use consistent units in all denominators.
- Reference issues: Record independently established expected issues, their severity and source. Explain adjudication and coverage limits.
- Correct findings: Count AI findings that match an independently confirmed reference issue using the predefined matching rules.
- False findings: Count generated findings that are unsupported or incorrect under the matching rules. Preserve their disposition.
- Missed reference issues: Count expected issues the AI failed to identify; retain the issue identity and severity rather than reporting only an aggregate.
- Precision: Calculate correct findings / (correct + false findings). Report counts and state not estimable when the denominator is zero.
- Recall: Calculate correctly identified reference issues / all reference issues. Report counts and state not estimable when there are no reference issues.
- Critical recall: Calculate detected critical reference issues / all critical reference issues. List each critical miss and its consequence; overall recall must not hide it.
- Key limit: State dataset size, representativeness, exclusions and other limitations that prevent generalising the observed results.
- Workflow concern: Describe how AI output could mislead or overload the reviewer, and what evidence is needed about the complete human-review workflow.
- Decision: Record the supported use or restriction, the evidence and unresolved issues behind it, and the accountable decision owner.
Define the task and unit of analysis before entering numbers. Record the reference-set method and protected evaluation boundary. Report subgroup results and limitations.
Are denominators clear? Are duplicate findings handled consistently? Are the cases independent enough for the analysis? Does the conclusion reflect missed consequential issues?
Completed fictional example
Fictional teaching example; replace its values with your own assessed facts and evidence.
Blank template
System / record ID: ____________________
Prepared by / date: ____________________
Required review / approval under local procedure: ____________________
Relevant regulations and guidance
These sources apply within their stated scope. The suggested worksheet fields, test designs and fictional decisions are practitioner recommendations, unless a source requirement is expressly identified. Completing a template does not establish compliance.
Regulatory guidance / GMP guideline · United States · February 2026
Scope: Medical-device manufacturers assuring software used in production or their quality management system.
Relevance: For medical-device production or QMS software, supports selecting assurance activities from intended use, risk and available evidence. It is nonbinding guidance and does not determine other sectors’ obligations.
Status: Final / published · Checked 2026-09-21
Publication identity, scope and status checked against the linked official source. This is not an independent audit of every retained guide statement.
Read source details and applicability →Regulatory guidance / GMP guideline · European Union · January 2011 revision
Scope: Computerised systems supporting activities in the applicable EU pharmaceutical GMP context.
Relevance: For computerised systems used in EU GMP activities, informs lifecycle, supplier and record controls. Confirm applicability to the actual activity; the example does not establish it.
Status: Final / published · Checked 2026-09-21
The Commission index confirms the January 2011 operative revision. The linked PDF could not be retrieved in this check; retained detailed section mappings need review against an authorized copy.
Read source details and applicability →Industry guidance / framework · United States · AI RMF 1.0 (2023); GenAI Profile (2024)
Scope: Cross-sector AI governance and risk management, including generative AI.
Relevance: A voluntary framework for describing AI risks, evaluation and ongoing management. Its suggested practices are not regulatory requirements.
Status: Final / published · Checked 2026-09-21
Publication identity, scope and status checked against the linked official source. This is not an independent audit of every retained guide statement.
Read source details and applicability →