The complaint arrived eleven months after the first customer was affected.
An AI system, including AI agents, had been applying a slightly wrong rule to a large volume of customer transactions. Mispricing, in breach of regulation. Every individual decision had looked reasonable.
The board reached for the incident protocol built after the cyber scare two years earlier. Contain. Notify. Investigate. Report back.
Trust most of that protocol. But six of the assumptions underneath it belong to cyber alone, and each one changes what the board does on the day.
One. That someone did this to you
Cyber hands you an attacker, and the response reaches for the language of being targeted. Those words do damage on a day like this.
This AI system was approved by this board and is earning money for this business. You're responsible for selecting it.
In a cyber event, being attacked goes some way to explaining why you didn't see it coming. Here, this company chose the system, and somebody was meant to be governing it.
The board settles now what this company says publicly while it still can't explain how its own system failed customers.
Two. That something anomalous is there to find
An intrusion leaves behind something that doesn't belong. An account active at three in the morning.
An AI system producing harm leaves the place tidy. Every operational check passes, every month, while the damage accumulates. The system is doing exactly what it was built to do.
- What would have to happen for a problem like this to reach us?
- Who in this company examines outcomes rather than systems?
Three. That you can disconnect and rebuild
Isolation works in cyber because it's temporary. Systems come down, the intruder goes, and you restore to a point before the compromise.
Here, the system that caused the harm is the system that does the work. Stopping it is real containment. The restore has no equivalent, because the only version to go back to is the thing you just switched off.
Two clocks run. One started eleven months ago. One started this morning. Some reporting deadlines attached to AI failures run in days.
A board that keeps the shutdown call for itself has decided the system keeps running, because no board convenes inside that window. Decide in advance which AI systems can actually be stopped, what stops with them, and who holds the authority to pull one at an agreed threshold.
Four. That the chain ends at a decision-maker
Cyber usually surfaces an act you can point to. A patch deferred. A credential handed over.
An AI system learns patterns from what it's been shown and applies them to new cases. Your customer base shifts. The provider updates the model underneath. What the system applies shifts with it. So there may be no decision to find. Nobody chose the rule that caused the harm, and it never came up for approval.
The evidence moves as well. The version that produced those outputs may be gone by the time anyone looks.
"We know it happened, and we can't tell you what caused it." That sentence is hard to work with in the boardroom. It's harder to say outside it.
Where a cause can be found afterwards, accountability can be settled afterwards. Where a cause may never be found, accountability gets settled in advance or never at all.
So one named executive owns the outcome of every AI system that matters to the business. A name on its own does very little. What makes it work is what that person secures before go-live: records deep enough to show how a decision was reached, the running version logged, the provider bound to preserve evidence and cooperate in an investigation into its own product, and the authority to stop the system. All of it is bought at the contract table.
The schedule of matters reserved for the board is where this belongs. Add one line. The board approves which AI systems require a named owner, and approves the names.
Five. That there's a fix you can prove
A cyber incident closes. You find the hole, shut it, and show that it's shut.
An AI system's behaviour under a given set of conditions is a property of that system. It produced those outputs once, so it will produce them again under those conditions. Non-recurrence becomes promisable once somebody can describe the conditions, which returns you to section four.
One decision belongs to the board. Where management can't describe what produced the failure, the system stays out of that application. It goes back when somebody can say what happened. Management will be under pressure to restore a capability the business has come to rely on, which is exactly why the call sits with you.
Six. That the cover will be there
Cyber insurance moved in three phases. Ordinary policies paid, because they said nothing about cyber. Then insurers wrote broad exclusions. Then a standalone market grew.
Boards remember the ending. The middle ran for years, and that's where the uninsured losses sat.
AI is in the middle phase now, and it's moving through this year's renewals. Standard wordings let insurers exclude losses connected to AI-generated output from general liability cover. Some cap AI-related recovery at a small fraction of the limit. The same language is reaching directors' and officers' cover.
Same company, same policy number, a different answer after renewal. Plenty of organisations will find out which one applies at the moment they claim.
It also changes arithmetic this board has already approved. A business case built while an insurer carried part of any loss reads differently once the whole loss sits with you. Put that beside the efficiency numbers from Edition 9.
Ask at your next renewal, with the answers in writing:
- Does any policy in our programme now exclude or limit AI-related loss?
- Does our directors' and officers' cover carry the same language?
- If an AI system caused a significant loss tomorrow, what would we actually recover?
Where this sits in the series
Governance Decoded Editions 7 and 8 built the vocabulary and the questions. Edition 9 showed a dependency on a single outside provider forming quietly while the board watched good news. Edition 10 asked where AI governance sits, and put the answer with the Chair. All of them assumed a board with time.
This edition works in the hours after the time has run out. It sharpens Edition 9, because the provider sitting inside your operations also holds the record of what its system did, and establishing what happened means asking that provider to help demonstrate that its own product caused the harm. It conditions Edition 10, because committees meet on a calendar, and deciding where AI governance sits has to include deciding who acts when nobody can be convened.
What changes for you personally
Good-faith reliance on management and on expert advice is your usual protection. That assumes the expert can explain the thing. Where the people who built a system can only partly account for its behaviour, you're relying on assurance alone.
Your duty of oversight is unchanged. The evidence you'd need to show that you discharged it has changed. That evidence is the record of what you asked, when you asked it, and what you were told.
So there's a habit worth forming. When an AI deployment comes to the board, ask who owns the outcome and what they secured at contract stage. See that the answer reaches the minutes. Where the name is still open, see that this reaches the minutes too.
Language / Clarity / Confidence.
Language. The sentence to carry is "who owns this outcome?" Owning an outcome and understanding a technology are two different things, and only one of them can be assigned to a person.
Clarity. Seek clarity on which parts of your incident plan were built around an attacker.
Confidence. You need the confidence to ask for a name, and to sit with the pause when none comes back. A silence is an answer. Record it.
Boardroom Takeaway
For the Chair: "Our incident plan assumes somebody attacked us. I want it tested against a failure we caused ourselves, and I want a named executive against every AI system that matters, appointed before contracts are signed."
For you as a board member: who owns this outcome, and what did they secure before we signed? And what would we recover if this went wrong?
Names get settled while the room is calm. Once the complaint arrives, the board is left looking for something that may never be findable.
If you're a CFO, a risk manager, or the person who signs off the insurance programme, the renewal in front of you may carry wording that moves a whole category of loss onto your own balance sheet. Put that in front of your board before the policy is bound, rather than after a claim.
Until next time,
- Paul