Incident reports and operational runbooks
Write a handover that another person can act on safely.
By the end of this lesson
- Explain incident reports and operational runbooks and when it is useful
- Perform a safe inspection or structured practice task
- Verify the result and describe remaining uncertainty
Your environment: Use an unprivileged account and a disposable Linux lab. Commands and available utilities vary by distribution and version. Where a command is unavailable or would require production access, complete the scenario as an offline written exercise. Setup help
Understand the concept
A runbook gives an operator a bounded procedure with prerequisites, decision points, verification and escalation. An incident report records impact, timeline, actions and current state. Useful documents are concise but specific: include service scope, timestamps, evidence references and what has not been checked.
Separate diagnostic steps from potentially disruptive remediation. Identify approval points, stop conditions and rollback. Do not include secrets in logs or handovers. After resolution, record contributing factors and follow-up owners without turning the review into blame.
Apply a careful workflow
A runbook should be tested and updated after real use. Confirm scope before consequential changes, keep a short record of expected and observed results, and consult local documentation when behaviour differs. Protect credentials and personal data in notes and logs.
Check your understanding
The goal of this lesson is not to memorise a single command. Explain why the proposed check is relevant, what its output can establish and what remains unknown. A successful command status is only one piece of evidence.
See it in context
Worked example
A bounded example to inspect in your own lab; adapt paths and options to your installed tools.
mkdir -p "$HOME/training4linux/runbook"
cat > "$HOME/training4linux/runbook/template.md" <<"EOF"
## Symptom
## Impact and scope
## Timeline (with timezone)
## Evidence and unknowns
## Actions and results
## Current state / next owner
## Verification and rollback
EOFRead each command and identify its target, inputs, expected evidence and possible failure modes. Do not run administrative or modifying commands on a production system.
Your turn
Practical lab
Complete the template for a fictional web service returning intermittent 503 responses. Include one next action, one stop condition and a clear handover owner.
Run commands in your own lab. This website displays instructions; the practice terminal below helps you rehearse the syntax.
1. Prepare your workspace
mkdir -p "$HOME/training4linux/extended-labs" && cd "$HOME/training4linux/extended-labs"
printf "%s\n" "Practice workspace for Incident reports and operational runbooks" > lab-notes.txt2. Complete the task
- Complete the scenario using the supplied synthetic information or a disposable local lab. Record what you observed and what you inferred.
- Compare your work against the success criteria and explain any differences.
- Write one verification step, one stop condition and a safe cleanup or recovery step.
3. Check your result
- The report separates facts from hypotheses and includes verification and escalation.
- The work remains within the disposable workspace or an offline case study.
- The report identifies at least one limitation or remaining uncertainty.
Need a hint?
Start with a read-only check, define the scope and choose the smallest next step that distinguishes competing explanations.
Reveal a worked solution
Compare this with your approach after trying the task. More than one valid solution may exist.
Use the success criteria as a review rubric. Record the reasoning and evidence behind your result rather than treating one exact command sequence as the only valid answer.After the lab: Retain your notes if useful. Remove only files created in the disposable practice directory; no production system should be changed.
Common mistakes to avoid
- Avoid writing a runbook that says only “restart the service” without preconditions or checks.
- Do not treat a successful command exit as proof that the overall service or business outcome is correct.
Check your understanding
Knowledge check
Answer all 3 questions correctly to pass. Retry as often as you need; your best score is kept.
Your lesson progress
Not started · Best score: 0% · Attempts: 0
Practical completion is self-reported. Progress is linked to a private cookie in this browser and stored on the site. Clearing that cookie loses access to it. Manage or erase progress.