Short lessons for familiar assurance questions
These lessons explain one decision at a time. The text below can be read independently. A recording should be added only when it exists, with captions and a matching transcript.
Video 1. The same training system can contain different risks
Spoken script
Imagine a training system with two features. One sends an email reminding you that training expires next week. The other tells the manufacturing system whether you are allowed to start a task.
Both features use training data, but ask what happens if each fails.
If the email is late, a supervisor’s overdue review and the manufacturing eligibility check may still prevent the operational problem. If the eligibility message is wrong, an operator with expired training may be allowed to start work.
That difference should affect the questions we ask and the evidence we collect.
For the reminder, we may check timing, recipients, and the information displayed. For eligibility, we need to examine identity mapping, expiry rules, time boundaries, missing responses, overrides, and what the receiving manufacturing system actually does.
Now change one assumption. Suppose the reminder email is the only control anyone uses before work starts. It is no longer just a convenience. Our assessment must follow that actual reliance.
The lesson is simple: do not assign the entire application one label and stop thinking. Describe each meaningful use, its failures, its dependencies, and the controls that matter.
On-screen example
| Reminder | Eligibility interface |
|---|---|
| Prompts renewal | Controls task start |
| Check recipient and timing | Check expiry, identity, response, and receiving action |
| Importance depends on other controls | Importance depends on operational reliance |
Viewer exercise
Choose a dashboard from your work. What action does someone take because of it? The answer may change how you assess that dashboard.
Video 2. Why a passing interface message can still be wrong
Spoken script
Your interface test returns “success.” Does that prove the business result is correct?
Consider a laboratory result moving from LIMS to ERP. The message arrives, the API returns a successful response, and the destination stores a number.
But what if the source uses milligrams per liter and the destination expects micrograms per liter? What if the number belongs to the wrong sample? What if an earlier status overwrites a later approval? What if a retry creates the same disposition twice?
A successful technical transaction answers only part of the question.
Start with the intended use. If ERP relies on the result to control material issue, test the source information, transformation, destination meaning, and resulting issue decision. Include units, identifiers, approval status, duplicates, delays, and recovery where they matter.
Keep evidence that connects the source transaction to the receiving record. A correlation identifier, a structured comparison, and an observed business action may tell the story better than a screenshot of a green icon alone.
The key question is not only “Did the message arrive?” It is “Did the right information arrive with the right meaning, and did the receiving process act correctly?”
Viewer exercise
List three fields whose wrong meaning could leave a message technically valid. Compare your answer with sample identity, units, and approval status.
Video 3. Why AI accuracy needs a denominator
Spoken script
Someone says an AI reviewer is 90% accurate. Before deciding whether that is useful, ask what was counted.
Were they counting documents, findings, sentences, or user ratings? Did the test include clean documents? Were difficult cases excluded? Were the most consequential misses visible?
Here is a fictional issue-detection example. There are 40 reference issues. The assistant finds 32 and misses 8. It also produces 6 false findings.
Its recall is 32 out of 40, or 80%. Its precision is 32 out of 38 proposed findings, or about 84%.
Now suppose 10 of the reference issues were critical and the assistant missed 2. The critical-issue recall is also 80%, but that result needs its own attention. The ordinary findings do not make those misses disappear.
The small test set also limits what we can infer about future performance. And the model result is still not the final human-review result.
Define the task, count consistently, report severity and uncertainty, and evaluate the combined workflow. A percentage becomes useful when you can explain its denominator and its relationship to the decision.
Viewer exercise
Would a shorter finding list necessarily be better? It could improve precision while reducing recall. Examine both and consider what users do with the result.