Monitoring plan
Record the approved intended use, model or service version, prompt and retrieval configuration, reference sources, supported population, usage volume, correct findings, false positives, false negatives, critical misses, reviewer overrides, incidents, user feedback, source changes, supplier changes, performance trend, acceptance criteria, corrective actions, stop criteria, and revalidation criteria.
Usage volume or engagement is not evidence that a GxP use remains acceptable. Measures must connect to the failure modes and decisions that were approved.
Stratify the measures
Report denominators and results by document or data type, issue type, criticality, language, format, site or population where relevant. Review every critical miss individually. Trend false findings because excessive noise can weaken human review. Evaluate overrides to determine whether they show healthy challenge, ambiguous criteria, model drift, source problems, or reviewer fatigue.
Completed fictional quarterly review
An AI assistant reviews validation protocols. Quarterly volume rises from 240 to 620 documents. Overall finding agreement improves, but critical-finding recall falls for scanned tables after a parser update. Reviewers accept more recommendations without opening the linked source. Two incidents involve obsolete procedure retrieval.
The validated-state conclusion is conditional. Scanned tables are removed from automated review, the parser and source index are corrected, reviewers receive focused retraining, and critical-miss and source-inspection thresholds are added. Revalidation covers the affected population and combined human-AI workflow before scope is restored.
Revalidation criteria
Reassess model or service version, prompt, reference corpus, retrieval settings, parser, guardrail, workflow, tool permissions, roles, supported population, acceptance criteria, material performance trend, critical miss, incident, supplier control, or monitoring-method changes.
Regulatory and procedural context
Primary sources. NIST AI RMF ↗ is voluntary and supports lifecycle AI risk management. Applicable GxP regulations, guidance, and approved validation procedures determine the regulated obligations for the specific use.
Company procedure. The organization’s approved validation, change-control, supplier-management, information-security, data-integrity, records-retention, and Quality approval procedures determine the required records, roles, and approval route.
CSVtoCSA practitioner interpretation. Monitoring must show continued control of the approved GxP use and the specific errors that matter; generic AI utilization metrics are insufficient. This is a recommended validation approach, not a statement that every listed activity is a direct regulatory requirement.