Independent AI test and evaluation
How much testing is enough for an AI-enabled system?
The test community has asked this for decades. AI makes it harder in three ways. Here is a working answer.
The test community has asked "how much testing is enough?" for decades. AI makes the question harder in three ways.
The system keeps changing
A traditional system passes developmental test, then operational test, and is fielded. An AI system gets retrained and updated, and a model update can change behavior without changing a line of the surrounding software. One pass/fail event at the start is not enough; the evaluation has to come back each time the model or the data changes.
The operating conditions are wide
Sensors, weather, terrain, document types and user habits all shift performance. You cannot test every combination, so the job is choosing the conditions that matter for the mission and saying plainly what was not tested.
The human is part of the system
If operators don't trust the output, they route around it, and a technically good model delivers nothing. Enough testing includes watching real users work with it.
A working answer
You have tested enough when a decision-maker can say, in one sentence, what the system can be trusted to do, under what conditions, and what would make you look again, and when that sentence holds up when checked a second, independent way.
Evaluation is the gate at every step of AI adoption, not a box at the end.
Further reading
- National Academies: Test and Evaluation Challenges in AI-Enabled Systems for the Department of the Air Force
- Institute for Defense Analyses: "How Much Testing Is Enough?" 25 Years Later
- The Department of War's AI T&E frameworks
If you're about to buy, field or renew an AI system and want an outside answer, let's talk.
Book a free 30-minute call