← All writing Essay 09

When AI is involved: seven questions your incident response process isn't asking.

AI incidents don't arrive through a separate channel. They arrive through your existing IRP. The problem is that your process has seven blind spots when the incident involves AI.

Rajesh Srinivasan 06 / 2026 11 min read governance · incident response

AI incidents don't arrive in a separate queue. They arrive the same way every other incident does, a customer complaint, a monitoring alert, a call from legal. The question is what your existing process does next.

Most organisations with mature engineering practices have invested significantly in incident response. Runbooks, on-call rotations, escalation paths, post-incident review templates, the discipline of responding to production failures is well-established. When a server goes down, the process starts. When a database is unavailable, the process starts. The team knows what to do.

The same process starts when an AI system produces an output that causes an incident. And that is precisely where the gap appears. The existing incident response process was built for infrastructure failures, software defects, and configuration errors. It was not built for the specific questions that an AI governance incident requires. The process itself is not the problem. Seven blind spots in the process are.

These seven questions do not replace the existing process. They layer onto it, the AI governance questions that need to be added to the incident response workflow that already exists, triggered when the incident involves AI output.

Read together, they are one set of questions: the human navigator's questions when the agent was driving. The agent drives. The human navigates. The seven questions assume a navigator who was, or should have been, in the seat, and can be asked them now.

When AI Is Involved: Seven Questions Your Incident Response Process Isn't Asking. Circular diagram showing seven questions around AI Incident Response centre: 1. Was AI involved? 2. Who was accountable? 3. What did it produce and for how long? 4. Preserve model state before remediation? 5. One-time error or systematic drift? 6. Regulatory notification? 7. What governance control failed? Your incident response process is the engine. These seven questions are the AI governance layer.
Essay 09 · Visual summary, seven questions, one layer
Your incident response process is the engine. These seven questions are the AI governance layer.

Question 01, Was AI involved in producing this output?

This is the triage question. It changes everything about what follows, which questions to ask, which evidence to preserve, which people to involve, and whether regulatory notification obligations apply. Standard incident response does not ask it.

Standard IRP asks. What system failed? Who owns it? What is the immediate impact?

AI governance layer adds. Was the output that caused this incident produced or influenced by an AI system? Which one? At which stage?

The answer to this question routes the incident down a different branch of the response process, one that asks different questions about accountability, evidence, and notification. Without this triage question, AI incidents get handled as generic software failures, and the governance-specific steps are never taken.

Question 02, Who was the named human accountable for this AI output?

Standard incident response asks who owns the service. That is an organisational question, which team, which on-call engineer, which engineering manager is responsible for this system's availability. It is the right question for an infrastructure incident.

An AI governance incident requires a different question: who is the named human who reviewed and attested to this AI output before it was acted upon? This is not an organisational ownership question. It is an accountability question, and the answer determines whether governance existed at the time the incident occurred.

Standard IRP asks. Who owns this service? Who is the on-call engineer?

AI governance layer adds. Who was the named human who reviewed and attested to this specific AI output? Can they be identified within 48 hours?

If the answer to the AI governance question is "nobody" or "we don't know", that is itself a governance finding, independent of whatever the original incident was. The absence of named accountability is not just a gap in the incident response. It is the incident.

Question 03, What did the AI produce, and for how long?

Infrastructure incident response measures blast radius in downtime: the system was unavailable for X minutes, affecting Y users. This metric maps cleanly to SLA obligations and business impact calculations. The recovery point is the moment the system came back online.

AI governance incidents have a fundamentally different blast radius. The AI system may have been producing incorrect, biased, or indefensible outputs for weeks before the incident was detected. The blast radius is not measured in minutes of downtime, it is measured in the number of outputs produced from the point the AI system began failing to the point the incident was detected.

Standard IRP asks. How long was the system down? How many users were affected during the outage window?

AI governance layer adds. From what point did the AI system begin producing problematic outputs? How many outputs were produced in that window? Who acted on them?

The detection gap. Infrastructure incidents are typically detected within minutes by monitoring systems. AI governance incidents are frequently detected by clients, regulators, or third parties, often weeks after the problematic outputs began. The blast radius may be orders of magnitude larger than the organisation assumed when the incident was first raised.

Question 04, Have you preserved model state before remediation?

Standard incident response follows a well-established sequence: detect, contain, preserve evidence, then remediate. In infrastructure incidents, evidence preservation typically means retaining logs, and remediating (restarting a service, rolling back a deployment) does not usually destroy the log record of what happened.

AI governance incidents require a different evidence preservation step, one that must happen before any remediation touches the AI system. Once the model is retrained, rolled back, or updated, the state that produced the problematic outputs may no longer be recoverable.

Standard IRP. Preserve system logs. Then remediate, restart, rollback, or patch as required.

AI governance layer adds. Before any remediation: preserve model version, input data sample, output record, review trail, and attestation records. Remediate only after evidence is secured.

Fix first, evidence gone. The most common mistake in AI governance incident response is prioritising remediation over preservation. Retraining or updating a model before capturing its state removes the evidence base needed for root cause analysis, regulatory response, and legal defensibility. The pressure to fix quickly is real, but the evidence window is short and not recoverable after the fact.

Question 05, Was this a one-time error or systematic drift?

Root cause analysis in standard incident response looks for a specific trigger: the deployment that introduced a bug, the configuration change that caused a failure, the dependency that became unavailable. The failure is acute, the trigger is identifiable, and the fix addresses the specific cause.

AI governance incidents have a different failure taxonomy. They may be acute, a specific prompt or input that produced a problematic output. Or they may be chronic, a model that has been drifting as data distributions shifted or environmental context changed, producing outputs that degraded gradually rather than failing suddenly. Standard RCA templates do not distinguish between these, and the remediation for each is completely different.

Standard RCA asks. What was the trigger? What change caused this failure? How do we prevent this specific failure from recurring?

AI governance layer adds. Was this acute (specific input) or chronic (systematic drift)? Root cause: model failure / data shift / environmental change / oversight gap?

The answer determines the remediation path. An acute error may be addressable with input validation or output guardrails. A chronic drift requires model re-evaluation, data pipeline audit, or re-training, and a monitoring infrastructure change to detect the same drift earlier next time.

Question 06, Does this trigger regulatory notification?

Most mature organisations have notification protocols for data incidents, GDPR breach notification within 72 hours, sector-specific obligations for financial or health data incidents. These checklists are well-understood and regularly rehearsed.

AI governance incidents introduce notification obligations that most existing checklists do not cover. The EU AI Act creates supervisory authority notification obligations for serious incidents involving high-risk AI systems. GDPR Article 22 creates specific obligations when automated decision-making affects individuals. These are not captured in the standard breach notification checklist.

Standard notification checklist. Was personal data involved? Was it a breach? GDPR 72-hour clock. Sector-specific obligations.

AI governance layer adds. Is this a high-risk AI system under the EU AI Act? Does it trigger supervisory authority notification? Were automated decisions affecting individuals involved? GDPR Article 22 obligations?

This question should be asked within the first 24 hours of an AI governance incident, not at the end of the investigation. The notification window for AI Act obligations may be shorter than the investigation timeline, and late notification typically compounds the regulatory exposure.

Question 07, What governance control failed, not what code failed?

Standard post-incident review asks what failed technically: which line of code, which configuration, which dependency. The output is a technical fix and a process improvement to prevent the same technical failure from recurring. This is exactly the right question for an infrastructure or software incident.

For an AI governance incident, the post-incident review needs to ask a different question in parallel: which governance control should have caught this before it reached production, and why didn't it?

Standard post-incident review. What was the technical root cause? What code or configuration change fixes it? How do we prevent this failure mode from recurring?

AI governance layer adds. Which gate should have caught this? Which attestation was missing or inadequate? Which oversight checkpoint did not hold? What changes to governance controls prevent this next time?

This question is the one that makes AI governance incidents generative rather than merely corrective. A technical fix prevents the specific failure from recurring. A governance fix addresses the structural gap that allowed the failure to reach production in the first place, and that structural gap, left unaddressed, will produce the next AI governance incident through a different failure mode.

The Navigator Framework connection. Organisations that have implemented the Navigator Framework phase gates enter the post-incident review with a named governance structure to audit against. Which gate was in place when the problematic output was produced? What did the Gate 3 (Build) probe record show? What did the Gate 4 (Verify) attestation cover? The framework provides the governance map that makes Question 7 answerable, and its answer actionable.


Adding the layer, not building a new process.

These seven questions are not a parallel incident response process. They are an AI governance layer that activates within the existing process when the incident involves AI output. The trigger is Question 1: was AI involved in producing this output? If yes, the layer activates. If no, the existing process runs as it always has.

In practice, this means two additions to the existing incident response infrastructure: a triage question added to the initial incident classification step, and an AI governance checklist, covering Questions 2 through 7, that activates when the triage question is answered affirmatively.

The checklist does not add significant overhead to incidents where AI is not involved. It adds significant structure to incidents where AI is, structure that determines whether the organisation can defend its response to a client, a regulator, or a board asking what it did and why.

The organisations that will handle AI governance incidents well are not the ones that build the most sophisticated separate processes. They are the ones that ask these seven questions before anyone else does, and have the governance infrastructure in place that makes the answers available when the questions arrive.

Read next
What regulators expect.
Essay 10 →
All writing