What Belongs in an Incident Response Plan
An incident response plan should define severity levels, who declares an incident, the roles during one, how to communicate, and where to find system-specific procedures. It is written for someone under pressure who has not read it before, which means it must be short, reachable without the affected systems, and exercised.
The parts that earn their place
How to declare, and who can. The most common failure is not a bad response, it is a late one, because people wait for certainty or for permission. Say explicitly that anyone can declare, that declaring wrongly carries no penalty, and give the exact mechanism: the command, the channel, the number. One line, near the top.
Severity levels defined by consequence and action. Levels are only useful if each maps to something concrete: who gets paged, whether customers are notified, whether an out-of-hours escalation is justified. Severity described with adjectives produces argument during the incident, which is the worst moment for it. Include one example per level from your own history.
Roles, minimally. An incident commander who coordinates and decides, a communications lead when the incident is customer-facing, and a scribe for anything lengthy. The critical rule is that the commander does not debug; the moment they are in a terminal, nobody is coordinating. For small teams one person can hold several roles as long as the roles are named and the debugging separation holds.
Communication channels, including the fallback. Where the incident is coordinated, and what to do when that system is part of the outage. A plan that assumes your chat platform works has a single point of failure in its first paragraph.
Escalation paths with names and numbers. Who to wake, in order, for each domain, and the vendor support path with the account identifier and the severity definitions the vendor uses. Looking that up during an incident wastes the window in which raising a case matters most.
When to involve legal, privacy, and leadership. Specifically for suspected data exposure, where notification clocks may start at the moment of awareness.
What to cut
Long documents do not get read under pressure, and everything that dilutes the usable parts makes them harder to find. Cut background on why incident management matters, definitions of common terms, organisational charts, tool overviews, and anything a reader could derive. Move system-specific runbooks out into their own documents and link them; the plan is the coordination layer, not the technical procedure. If your plan is over about four pages, the parts that would be used at 3am are competing with the parts that never will be.
The parts people forget
Reachability when things are broken. If the plan lives in a wiki behind the identity provider that has failed, it does not exist. Keep a copy somewhere independent: a different provider, an offline copy on responders' machines, or print. The same applies to contact lists and vendor account numbers.
Handover between responders. Long incidents outlast people. A defined handover, covering current state, what has been tried, what was ruled out, and open threads, prevents the second shift repeating the first shift's work. This is where a scribe pays for itself.
Explicit permission to take disruptive action. Responders hesitate over failing over, rolling back, or disabling a feature because the action feels drastic. Stating in advance which actions are pre-authorised at which severity removes a delay that is otherwise paid in full every time.
What to preserve before restarting. Restarting a process destroys the evidence needed to explain the failure. A short list of what to capture first, such as logs, memory state, or a snapshot, preserves the ability to run a useful postmortem without delaying recovery much.
Suspected security incidents diverge early. If compromise is suspected, the priority order changes: containment and evidence preservation matter alongside restoration, and actions that would be sensible for a fault, such as wiping and rebuilding a host, can destroy the only record of what happened.
Third-party incidents. Outages you cannot fix and breaches at vendors are different playbooks, and both are common enough to warrant a paragraph each.
How the incident ends. Criteria for closure, who decides, and what has to be true: service confirmed restored, backlogs drained, follow-up actions recorded.
Keeping it real
Exercise it, cheaply and often. A thirty minute tabletop walking a plausible scenario finds broken contact details, missing permissions, and unclear ownership faster than any review. Run one per quarter and change the plan afterwards. A plan that has never been exercised is a document rather than a capability.
Update it from real incidents. Every incident produces evidence about the plan: what could not be found, which step was skipped, who was paged unnecessarily. Making a plan amendment a standard postmortem output is what keeps it current without a separate review cycle.
Verify the on-call data specifically. Phone numbers, escalation lists, and vendor account details rot silently and are needed precisely when they cannot be looked up. A scheduled check of exactly those fields catches the most consequential kind of staleness.
Measure time to declare. Detection to declaration is usually the largest recoverable delay in an incident and the least examined. If it is long, the cause is normally uncertainty about permission, which the plan can fix directly.
Keep one owner. A document with no owner drifts until it is wrong, and a plan that is confidently wrong is worse than an obviously outdated one, because people follow it.
Frequently asked questions
- What should an incident response plan contain?
- How to declare an incident and who may, severity levels tied to concrete actions, the roles during a response, communication channels including a fallback, escalation paths with names and numbers, when to involve legal and leadership, and links to system-specific runbooks kept as separate documents.
- How long should an incident response plan be?
- Short enough to be used under pressure, which in practice means a few pages. Everything that will never be read at 3am competes with the parts that will, making them harder to find. Background, definitions, and tool overviews belong elsewhere, and technical procedures belong in runbooks.
- Who can declare an incident?
- Anyone, and the plan should say so explicitly along with the exact mechanism. The most common response failure is lateness, caused by people waiting for certainty or permission. Stating that declaring wrongly carries no penalty removes the hesitation that costs the most time overall.
- Should the incident commander also debug?
- No. The moment the commander is in a terminal, nobody is coordinating, tracking the timeline, or communicating. In a small team one person may hold several roles, but the separation between coordinating and debugging is the one that should not be collapsed, because both degrade when combined.
- How often should an incident response plan be tested?
- A short tabletop exercise each quarter is enough to find broken contact details, missing permissions, and unclear ownership. Real incidents should also feed amendments back as a standard postmortem output, which keeps the plan current without needing a separate review process.