Monitoring should lead to a decision
Choose indicators tied to meaningful failure: confirmed missed issues, false findings, wrong sources, unauthorized retrieval, reviewer overrides, tool failures, and unsupported use. Assign owners, review frequencies, thresholds or event triggers, and actions appropriate to the process.
A stable average score can hide a serious failure in one document type. Report relevant subgroups and investigate changes in case mix before interpreting trends.
Worked incident — fictional
A source refresh fails, and the assistant continues answering from an obsolete procedure set. A user reports a conflicting instruction.
The owner restricts affected use, identifies the period and outputs potentially affected, evaluates downstream decisions, restores the correct sources, and investigates why monitoring did not detect the failed refresh. Corrective actions address the actual cause and their effectiveness is checked. Not every issue automatically requires the same CAPA process; follow the applicable quality procedure and impact assessment.
Completed monitoring rule
A confirmed unauthorized-source disclosure triggers immediate incident escalation and restriction of the affected retrieval path. Wrong-version findings trigger assessment of source synchronization, affected outputs, and continued use. The owner records the decision and required recovery evidence.
Exercise
The tool’s overall rejection rate falls. Does that prove improvement?
Show suggested answer
It could reflect better output, easier cases, less careful review, or changed user behavior. Investigate with supporting evidence before claiming improved performance.
Expected records
A monitoring plan, incident/deviation records, CAPA where appropriate, periodic review and decisions about continued use. Adapt record names and detail to the applicable procedures; these are practitioner examples, not a universal mandatory document list.
Relevant regulations and guidance
Regulations set requirements within their scope; guidance offers recommendations or explains applicable expectations; standards depend on adoption or incorporation. Voluntary frameworks do not establish compliance. The lesson's controls, test designs and example records are practitioner suggestions unless explicitly identified otherwise.
Guidance · International Council for Harmonisation (ICH); regional implementation by FDA and other authorities · R1; January 2025 typographical correction
Scope: International · Quality risk management across the pharmaceutical product lifecycle.
Relevance to this lesson: Pharmaceutical quality-risk management can inform proportionate review of changes and uncertainty.
Publication status: Final / published. Source checked 21 September 2026.
Publication identity, scope and status checked against the linked official source. This is not an independent audit of every retained guide statement.
Source details and applicabilityVoluntary framework · National Institute of Standards and Technology · NIST AI 100-1 · January 2023
Scope: United States · Cross-sector AI governance and risk management, including generative AI.
Relevance to this lesson: Supports ongoing risk management and response; pilot monitoring intervals remain local examples.
Publication status: Final / published. Source checked 21 September 2026.
Publication identity, scope and status checked against the linked official source. This is not an independent audit of every retained guide statement.
Source details and applicability