Who’s watching the bot? Major incident discipline, applied to AI.

Picture this: your AI assistant tells hundreds of employees the wrong way to fix a common problem. How long until someone notices?

If the honest answer is “when the tickets pile up,” you don’t have AI governance. You have hope.

I’ve served on the Major Incident Board for a 24x7 hospital system, attending about 10 major incident calls a week and rotating as Manager on Duty. In that world, nothing reaches production without change control, and every major incident gets an owner, a timeline and a root cause.

AI deserves the same discipline. Usually it gets less.

What watching the bot looks like in practice:

1. Treat AI changes like changes. New intents, workflows and knowledge sources go through change management, with a rollback plan.
2. Sample the answers. Someone reviews a set of AI conversations every week for accuracy, tone and missed escalations.
3. Give AI failures a severity. A bot confidently giving wrong answers at scale is a major incident. Run it like one.
4. Keep a human one click away. No user should ever be trapped in a loop with a machine.

The service desk’s new job isn’t only answering questions. It’s supervising the thing that answers them.

Next in the series: why you should consolidate before you automate.

CIOs and ITSM leaders: does your AI go through change management today? Be honest.