Data Governance, Provenance, and ALCOA+ for AI

In AI-enabled GxP systems, data is not merely an input. It shapes model behavior, defines the evaluated population, supports the validation conclusion, and may become part of the regulated evidence. This chapter applies data-integrity thinking across the AI lifecycle.

Published: September 4, 2026 | Version 1.0

Editorial owner: CSV to CSA Knowledge Hub | Review status: Open for practitioner peer review

THE DATA ASSURANCE CHAIN

Source → acquire → transform → label → curate → split → train → evaluate → deploy → monitor → retain.

Every transition needs an owner, version, rule, quality check, and recoverable evidence appropriate to risk.

APPLYING ALCOA+ TO AI DATA

Attributable

Identify source system, creator or device, extraction job, labeler, reviewer, transformation code, and approval. For automated data, attribution includes service identity and configuration.

Legible

Retain data and metadata in a form that can be interpreted. An embedding vector without the source, method, and index version is insufficient evidence.

Contemporaneous

Capture time, sequence, acquisition conditions, and delayed entry. Time matters for drift analysis and causal review.

Original or true copy

Define the authoritative source and control copies. A training extract should be traceable to the source snapshot and transformation.

Accurate

Check values, labels, joins, units, mappings, and transformations. Accuracy includes freedom from silent corruption and systematic labeling error.

Complete

Include relevant records, exclusions, failures, metadata, and rejected inputs. Do not retain only the data that successfully traversed the pipeline.

Consistent

Control definitions, units, timestamps, identifiers, ordering, and referential integrity across sources.

Enduring

Preserve the datasets, code, metadata, and evidence for the required period in a readable and controlled format.

Available

Retrieve the exact dataset and context needed to reconstruct evaluation or investigate an outcome.

DATA-PROVENANCE RECORD

For each dataset, capture:

• Dataset ID and version
• Purpose: training, tuning, validation, independent test, challenge, monitoring, or investigation
• Source systems and extraction window
• Inclusion and exclusion criteria
• Population, site, product, instrument, language, and time coverage
• Data owner and permitted use
• Transformations and code version
• Missing data and imputation
• Label definition, labeler qualification, disagreement resolution, and uncertainty
• Deduplication and leakage controls
• Quality checks and exceptions
• Split method and independence
• Storage, access, retention, and deletion
• Known limitations and approved use

LABEL QUALITY

Labels may be more uncertain than the model output. Establish a label ontology, decision rules, training, blinded review when appropriate, inter-rater agreement, adjudication, and a method for ambiguous cases.

Do not force uncertain examples into a clean binary class merely to simplify training. Preserve uncertainty or create an indeterminate class when the process supports it.

DATA LINEAGE TESTS

• Reconcile source counts to extracted counts
• Verify unique identifiers and duplicate handling
• Test joins for orphan, duplicate, and mismatched records
• Challenge units, time zones, encoding, and null values
• Compare transformation output with independently calculated samples
• Confirm inclusion and exclusion logic at boundaries
• Verify that withdrawn, superseded, or unauthorized records are excluded
• Reproduce the dataset from controlled source and code
• Confirm the deployed feature pipeline matches the evaluated pipeline

WORKED EXAMPLE: VISUAL INSPECTION DATA

A model classifies vial images as acceptable or defective.

Data risks:

• One camera or lighting setup dominates training data
• Images from the same physical unit appear in training and test sets
• Defect labels reflect inspector disagreement
• Cropping removes contextual evidence
• Synthetic defects do not represent real process failures
• Rare critical defects are underrepresented
• Image compression differs between training and production
• New container artwork changes the background distribution

Controls:

• Split by production lot, time, or physical unit—not random image alone
• Stratify by camera, line, product, defect type, and operating condition
• Retain original image and preprocessing trace
• Use qualified experts and adjudication for difficult labels
• Maintain a rare-defect challenge set outside development
• Test the deployed capture and preprocessing chain
• Monitor image-quality and feature-distribution indicators

RAG SOURCE GOVERNANCE

For retrieval-augmented generation, define:

• Which repositories and document states are allowed
• How approval, effective date, supersession, and access rights flow into the index
• Chunking and metadata rules
• Re-index frequency and completeness reconciliation
• Deletion and legal-hold behavior
• Retrieval authorization at query time
• How the user reaches the authoritative source

An approved SOP can still be retrieved incorrectly if a superseded chunk remains in the index.

PRIVACY AND CONFIDENTIALITY

Determine whether data may be transmitted to the provider, used for service improvement, retained in logs, or accessed by subprocessors. Apply minimization, de-identification, contractual restrictions, encryption, access control, and approved regions. Validation does not substitute for privacy or confidentiality review.

DATA CHANGE CONTROL

Assess changes to source systems, extraction logic, label rules, population, site mix, product mix, units, preprocessing, missingness, and retention. Some data changes require model reevaluation even if model code does not change.

PRACTITIONER REVIEW QUESTIONS

1. Can the exact test set be reconstructed?

2. Are labels reliable enough to judge the model?

3. Could one entity appear in more than one split?

4. Does production preprocessing match development?

5. Which data shifts would invalidate acceptance?

6. Can a regulated source be removed from every downstream index and copy?

PROFESSIONAL INTERPRETATION

Data integrity for AI is lifecycle integrity. The evidence must connect original source, controlled transformation, evaluation, deployed behavior, and monitored outcomes. A model card without dataset lineage cannot support a strong assurance argument.

PRIMARY SOURCES

MHRA, GxP Data Integrity Guidance and Definitions:

www.gov.uk/government/publications/guidance-on-gxp-data-integrity

FDA and EMA, Guiding Principles of Good AI Practice in Drug Development:

www.fda.gov/about-fda/artificial-intelligence-drug-development/guiding-principles-good-ai-practice-drug-development

EMA, Reflection paper on AI in the medicinal product lifecycle:

www.ema.europa.eu/en/use-artificial-intelligence-ai-medicinal-product-lifecycle-scientific-guideline

NIST AI RMF:

www.nist.gov/itl/ai-risk-management-framework