AI Assurance Academy · Part 2
Governance, Accountability, and the AI Inventory
Chapter 5 of 20 · AI governance becomes operational only when every production use has an owner, an identifiable configuration, a risk decision, a monitoring obligation, and a controlled path to change or retirement. This chapter turns policy into lifecycle gates.
AI governance becomes operational only when every production use has an owner, an identifiable configuration, a risk decision, a monitoring obligation, and a controlled path to change or retirement. This chapter turns policy into lifecycle gates.
Published: September 4, 2026 | Version 1.0
Editorial owner: CSV to CSA Knowledge Hub | Review status: Open for practitioner peer review
GOVERNANCE PRINCIPLE
Avoid creating a separate AI bureaucracy disconnected from the quality system. Extend existing governance—computerized systems, suppliers, data, security, privacy, change, incident, records, training, and CAPA—while adding controls for AI-specific uncertainty.
ACCOUNTABLE ROLES
Business process owner
Owns the intended use, process controls, benefit, users, operating procedure, and acceptance of process residual risk.
System owner
Owns the configured application, availability, access, integrations, support, inventory, and technical lifecycle.
Model owner
Owns model identity, performance, limitations, evaluation, versioning, monitoring, retraining or replacement, and technical residual risk.
Data owner or steward
Owns source legitimacy, definitions, quality, lineage, access, retention, representativeness, and correction.
Quality assurance
Provides independent challenge, approves the assurance approach when required, confirms issue disposition, and evaluates continued validated state.
Privacy and legal
Assess permitted data use, notices, agreements, cross-border transfer, intellectual property, and applicable AI law. These responsibilities cannot be replaced by a validation protocol.
Cybersecurity
Owns threat modeling, secrets, identity, access, supply-chain security, abuse monitoring, and response.
Human-factors or user-experience specialist
Evaluates whether users understand outputs, uncertainty, evidence, escalation, and override without automation bias.
Supplier
Provides lifecycle controls, documentation, change notice, incident support, security evidence, known limitations, and service commitments.
The same individual may perform several roles in a small organization, but decision rights and independence should remain clear.
THE MINIMUM AI INVENTORY
Identity
System name, use-case ID, owner, status, environment, supplier, and integration map.
Purpose
Intended use, prohibited use, users, process, products, sites, languages, and decision authority.
Technology
Model family, model/version identifier, hosting, training status, prompt/configuration, retrieval, tools, data flows, and deterministic controls.
Regulatory and records
GxP classification, governing process, electronic records/signatures, retention, privacy classification, and product-software distinction.
Risk and evidence
Risk conclusion, key failure modes, validation status, approved operating envelope, unresolved limitations, and residual-risk owner.
Operations
Monitoring metrics, thresholds, review frequency, last review, supplier notification channel, fallback, incident link, and retirement plan.
Inventory entries should be machine-readable enough to identify shared dependencies. If one foundation model changes, the organization should know every affected use.
LIFECYCLE GATES
Gate 0 — Experiment approval
Use synthetic, public, or approved de-identified data. No regulated decision or production action. Define owner, time limit, and exit criteria.
Gate 1 — Use-case qualification
Approve intended use, value, scope, preliminary risk, data availability, and prohibited use. Decide whether AI is necessary or a deterministic method is better.
Gate 2 — Design approval
Approve architecture, supplier, data strategy, controls, human oversight, security, privacy, records, evaluation plan, and monitoring design.
Gate 3 — Release approval
Review implemented configuration, traceable evidence, test results, issues, training, procedures, operational readiness, and residual risk.
Gate 4 — Continued-use review
Assess monitoring, drift, overrides, incidents, supplier changes, access, data shift, complaint or deviation signals, and benefit. Continue, restrict, change, suspend, or retire.
Gate 5 — Retirement
Disable access and agents, preserve records and metadata, export data, revoke keys, close supplier access, archive evidence, and verify downstream transition.
AI GOVERNANCE BOARD DECISION PACK
Keep the pack short enough to support a real decision:
- One-page intended use and process diagram
- Benefit and non-AI alternative
- Top failure chains and proposed controls
- Data and supplier assessment
- Evaluation and acceptance strategy
- Human oversight and user study
- Monitoring and change triggers
- Open issues, residual risk, and named owners
The board should challenge assumptions rather than approve a large document bundle.
WORKED EXAMPLE: ENTERPRISE LLM ADOPTION
The company licenses a private enterprise LLM. Twenty teams request access. Treating the license as one validated system would hide use-specific risk.
Create:
- A platform-level assessment for hosting, security, privacy, supplier change, logging, model options, and prohibited data
- A reusable control library for authentication, approved models, retention, and monitoring
- A use-case record for each GxP workflow
- Risk tiers that determine evidence, human oversight, and approval
- Usage analytics to detect unregistered applications
A meeting-summary use may rely mostly on platform controls. A complaint-triage use needs representative evaluation, process integration, monitoring, and accountable approval. An agent that changes quality records needs tool-level permissions and human gates.
GOVERNANCE METRICS
Useful leading indicators:
- Percentage of production uses with current owner and intended use
- Supplier/model changes assessed within target time
- Monitoring alerts reviewed on time
- Unregistered use detected
- Overrides and disagreements analyzed
- Open high-risk limitations and aging
- Models approaching review or retirement date
- Users trained before access
Avoid vanity metrics such as total AI projects or documents produced.
ESCALATION TRIGGERS
- Performance limit breached
- New serious or previously unknown failure
- Use outside approved scope
- Silent supplier or model change
- Data leakage or security event
- Human-review control shown ineffective
- Material subgroup disparity
- Inability to reconstruct a decision
- Agent executes or attempts an unauthorized action
PROFESSIONAL INTERPRETATION
Good governance reduces friction for low-risk experimentation and increases control as authority and consequence rise. The best inventory is not a spreadsheet that is updated before audits; it is the dependency map that drives change, monitoring, and incident response.
PRIMARY SOURCES
FDA and EMA, Guiding Principles of Good AI Practice in Drug Development:
NIST AI Risk Management Framework and Playbook:
www.nist.gov/itl/ai-risk-management-framework
EMA, Reflection paper on AI in the medicinal product lifecycle:
www.ema.europa.eu/en/use-artificial-intelligence-ai-medicinal-product-lifecycle-scientific-guideline
ISPE, GAMP Guide: Artificial Intelligence. Industry guidance; not a regulation:
ispe.org/publications/guidance-documents/gamp-guide-artificial-intelligence