Each important claim needs a basis
An evidence plan connects a claim to a test, review, analysis, or operational control. Avoid collecting only evidence that the application runs or that users like it.
Review evidence quality
Check whether the tested configuration matches the proposed release, the expected answers were established adequately, test failures remain visible, and the evaluation set was protected from development feedback. Identify where results are descriptive rather than strong evidence of general performance.
Exercise
Users report that the assistant “feels accurate.” What evidence gap remains?
Show suggested answer
Satisfaction does not quantify missed findings, unsupported statements, source errors, or final decision quality. Use user feedback to identify concerns, then evaluate the relevant behaviors directly.
Expected records
A validation evidence plan with claims, requirements, risks, evaluation methods, acceptance basis, owners and traceability. Adapt record names and detail to the applicable procedures; these are practitioner examples, not a universal mandatory document list.
Relevant regulations and guidance
Regulations set requirements within their scope; guidance offers recommendations or explains applicable expectations; standards depend on adoption or incorporation. Voluntary frameworks do not establish compliance. The lesson's controls, test designs and example records are practitioner suggestions unless explicitly identified otherwise.
Guidance · U.S. Food and Drug Administration · February 2026
Scope: United States · Medical-device manufacturers assuring software used in production or their quality management system.
Relevance to this lesson: In its production/QMS scope, informs risk-based assurance activities and records.
Publication status: Final / published. Source checked 21 September 2026.
Publication identity, scope and status checked against the linked official source. This is not an independent audit of every retained guide statement.
Source details and applicabilityVoluntary framework · National Institute of Standards and Technology · NIST AI 100-1 · January 2023
Scope: United States · Cross-sector AI governance and risk management, including generative AI.
Relevance to this lesson: Supports selecting measures and documenting unmeasured limitations.
Publication status: Final / published. Source checked 21 September 2026.
Publication identity, scope and status checked against the linked official source. This is not an independent audit of every retained guide statement.
Source details and applicabilityEditorial information and limitations
Educational chapter · Editorial version 2.0. The update date records substantive site editing; a completed independent technical review is not claimed.
AI tools assisted drafting and structural editing. Examples and results labelled fictional or synthetic are teaching material. Adapt the methods to the actual intended use, applicable requirements and approved procedures.
Estimated reading time uses the written lesson at 220 words per minute, with links and Markdown formatting removed. Video time comes from the media file and is separate.
Sources, authorship and editorial policy