Practical software assurance playbook
Validating AI-Enabled GxP Systems
A controlled path from context of use and human authority to evaluation evidence, release boundaries, monitoring, and stop conditions.
Professional interpretationEducational practitioner method · Quality review required for case-specific use
Validation process
Apply the lifecycle controls
- 01
Intended Use
- Assessment
- Specify the users, population, data, AI task, output, decision influenced, environment, exclusions, and prohibited actions.
- Expected record
- Feature-level intended use with permitted and prohibited reliance.
- 02
GxP Assessment
- Assessment
- Identify the regulated process, record, calculation, control, or decision affected by the AI-enabled use.
- Expected record
- Documented GxP applicability, scope, exclusions, and accountable owner.
- 03
Part 11 and Records
- Assessment
- Determine electronic-record and electronic-signature reliance from the predicate-rule record and configured workflow.
- Expected record
- Record inventory, Part 11 rationale, retention, accurate-copy, signature, and audit-trail requirements.
- 04
AI Role
- Assessment
- Determine whether the model retrieves, extracts, predicts, drafts, recommends, decides, or executes.
- Expected record
- Authority map showing deterministic, AI, and human responsibilities.
- 05
Decision Authority
- Assessment
- State which outputs inform a person, change process state, create a record, or execute an action, and which decisions remain prohibited.
- Expected record
- Approved decision-authority matrix with confirmation, escalation, and stop controls.
- 06
Supplier and Service Assessment
- Assessment
- Define the supplied model or service, release controls, available evidence, customer responsibilities, change notice, data terms, and known limitations.
- Expected record
- Supplier assessment with accepted evidence and customer-controlled gaps.
- 07
Human Oversight
- Assessment
- Define reviewer competence, time, source visibility, independence, escalation, and the consequences of automation bias.
- Expected record
- Oversight procedure and representative-user challenge results.
- 08
Failure Modes
- Assessment
- Trace omission, unsupported content, wrong source/version, subgroup failure, instability, prompt injection, overreliance, and action errors.
- Expected record
- failure scenarios linked to controls and critical acceptance criteria.
- 09
Evaluation Dataset
- Assessment
- Represent the approved population, critical edge cases, realistic prevalence, source quality, subgroups, and prohibited contexts.
- Expected record
- Versioned dataset specification, provenance, exclusions, and leakage review.
- 10
Expected Results and Acceptance Criteria
- Assessment
- Predefine correct behavior, permitted variation, unacceptable events, denominators, thresholds, confidence, reviewer recovery, and the action taken when criteria fail.
- Expected record
- Approved evaluation protocol with objective acceptance and stop rules.
- 11
False Positives, False Negatives, and Critical Misses
- Assessment
- Measure error by issue type and consequence; examine every critical miss instead of relying on one aggregate accuracy score.
- Expected record
- Confusion matrix, precision, recall, critical-event review, subgroup results, disposition, and retest traceability.
- 12
Source Verification and Data Integrity
- Assessment
- Verify source identity, version, provenance, completeness, transformation, retrieval, citation, retention, and reconstruction of inputs and outputs.
- Expected record
- Source-grounding tests, data-flow record, lineage, integrity controls, and accurate-copy evidence.
- 13
Controls
- Assessment
- Constrain authority through source filters, citations, deterministic gates, permissions, confirmation, logging, and kill switches.
- Expected record
- AI use and control requirements with tested preventive and detective measures.
- 14
Access, Audit Trail, and Recovery
- Assessment
- Restrict input, output, configuration, approval, and tool permissions; retain reconstructable activity and a tested safe fallback.
- Expected record
- Role challenge, audit-trail review, record reconstruction, failure handling, and recovery evidence.
- 15
Release Decision
- Assessment
- State the supported use, open uncertainty, conditions, fallback, authorized approver, and restart criteria.
- Expected record
- Signed release narrative and reconstructable evidence package.
- 16
Monitoring
- Assessment
- Detect changes in data, use, supplier model, prompts, retrieval, performance, user response, and critical-event rate.
- Expected record
- Monitoring plan with thresholds, owners, stop conditions, and reassessment criteria.
- 17
Change Control and Revalidation
- Assessment
- Assess model, prompt, retrieval, data, interface, permission, supplier, and intended-use changes before continued use.
- Expected record
- Change-impact decision with regression, restriction, revalidation, rollback, or no-action rationale.
- 18
Periodic Review and Retirement
- Assessment
- Confirm that use, controls, evidence, monitoring, access, supplier status, records, and assumptions remain current; plan controlled withdrawal when they do not.
- Expected record
- Periodic-review conclusion, open actions, reassessment criteria, and retirement or record-migration plan.
Release gate
Before the decision is defended
- Critical errors are defined in operational terms and independently challenged.
- Human review is designed and tested as a control rather than assumed.
- The approved approved conditions of use and prohibited actions are explicit.
- Monitoring can locate affected records and trigger suspension or reassessment.
Scope and limitations
What this playbook does not establish
- A high aggregate accuracy score is not a release conclusion.
- This playbook does not replace clinical, safety, privacy, security, or legal evaluation.
- Model and supplier opacity may require narrower use, stronger controls, or a no-release decision.
Use applicable regulations, final guidance, organizational procedures, and subject-matter review for the actual system and jurisdiction.