The AI You Validated Is Not the AI You’re Running
The AI You Validated Is Not the AI You’re Running
Somewhere in your business, software is making calls for a person used to make forecasting demand, routing trucks, quoting customers, and processing documents. At some point, someone looked at that tool and decided it could be trusted.
My question is simple: is it still the same tool?
The AI inside your software changes on the vendor’s schedule, not yours. Updates ship, behaviour shifts, and the documentation that justified your decision stays exactly where it was. Nothing tells you that the validation you did last year no longer holds.
If an employee changed how they made decisions and told no one, you would treat it as a performance problem. When software does it, we call it a product update and move on.
I learned this before AI made it worse
For twelve years I built and ran the traffic forecasts at Canada’s largest airport. In early 2020, every forecasting model in aviation failed at once, not because the models changed, but because the world under them did. For a while, organizations kept acting on numbers whose assumptions were already dead.
What I took from that: trust in an analytical system doesn’t last. It has to be re-earned when conditions change. AI adds a twist. Now it isn’t the only world that moves. The system moves too, and you don’t see it happen.
What we told the government
In September, Ainfore submitted a formal response to the Government of Canada’s consultation on AI transparency, the input stage for whatever rules come next. Most of the voices in that process build AI or regulate it. We wrote for the businesses that run it: mid-market operators who buy AI rather than build it and carry the consequences either way.
Our argument: transparency that only describes a system at launch isn't real transparency — because that's not when trust breaks down. Trust breaks down later, quietly, when the system changes, and no one tells you.
The fix we proposed fits on one page. Every AI product should come with a plain-language deployment summary: what it’s for, what it should not be used for, where it breaks, what human oversight the vendor expects, and how you’ll be told when it changes. Written for the person accountable for the output, not for another developer. Behind it, one rule with no exceptions: obligations sit with developers; benefits flow to deployers. A mid-market business shouldn’t have to fill out a single new form to get this.
👉 Read the full submission (PDF)
We ask nothing we don’t do ourselves
Ainfore keeps a deployment summary for what we ship, and we tell clients when it materially changes. If a firm, our size can make that commitment; no vendor has an excuse.
A question for your own operation
Canada’s rules will take time. Your tools are running today. So, pick one, demand planning, routing, whatever touches customers, and ask when trust in it was last re-earned.
If you don’t know, that’s normal. It’s also where our work usually starts.