About
Where this judgment comes from
AI test and evaluation since 2007: sensors and imagery first, then the Pentagon's AI programs.
I manage AI test and evaluation programs for the Pentagon. For nearly two decades I've answered one question for senior leaders: does this AI work well enough to trust? When the answer is no, the reason is usually the data or the workflow, not the model.
I'm a service-disabled veteran, and I lead AI Eval Corporation, which does this work for the Department of War. I hold a TS/SCI clearance, so I can work where the systems and the data actually live.
Timeline
- Army Research Laboratory. AI testing for sensor and imagery systems, 2007 to 2019.
- Project Maven. Former AI test and evaluation program manager, 2019 to 2023.
- OUSD(R&E) and the Chief Digital and AI Office (CDAO). I manage AI test and evaluation programs for the Pentagon, 2023 to present.
- AI Eval Corporation. President, 2023 to present.
What the work has taught me
Test what was delivered first.
Before any deep evaluation, confirm the thing in front of you is the thing that was promised.
Augment the job before replacing it.
AI earns trust by helping people do the work, then takes on more as the evidence comes in.
Systems learn the metric you give them.
When one number decides the contract, the system gets better at the number. Look at the work.
AI has to fit the workflow.
A capable model bolted onto the wrong process still fails. The workflow is part of what gets tested.
Judge competitors the same way.
When vendors compete, the evaluator's only job is a fair, repeatable comparison with no stake in the winner.
How I judge
I compare what was asked for with what was delivered, check it at least two different ways, and give a plain conclusion with the reasoning. No pile of scores to decode.
Elsewhere
If you're about to buy, field or renew an AI system and want an outside answer, let's talk.
Book a free 30-minute call