Runbooks Ops
Skill Profile
(Select at least one profile to enable specific modules)
Overview
Runbooks are documented procedures for operational tasks, incident response, and troubleshooting. Essential for maintaining reliable systems and enabling team members to handle issues independently.
Why This Matters
- Faster resolution: Step-by-step guides reduce MTTR
- Consistency: Same procedure every time
- Knowledge sharing: Reduce bus factor
- Onboarding: New team members can handle ops tasks
Core Concepts & Rules
1. Core Principles
- Follow established patterns and conventions
- Maintain consistency across codebase
- Document decisions and trade-offs
2. Implementation Guidelines
- Start with the simplest viable solution
- Iterate based on feedback and requirements
- Test thoroughly before deployment
Inputs / Outputs / Contracts
- Inputs:
- <e.g., env vars, request payload, file paths, schema>
- Entry Conditions:
- <Pre-requisites: e.g., Repo initialized, DB running, specific branch checked out>
- Outputs:
- <e.g., artifacts (PR diff, docs, tests, dashboard JSON)>
- Artifacts Required (Deliverables):
- <e.g., Code Diff, Unit Tests, Migration Script, API Docs>
- Acceptance Evidence:
- <e.g., Test Report (screenshot/log), Benchmark Result, Security Scan Report>
- Success Criteria:
- <e.g., p95 < 300ms, coverage ≥ 80%>
Skill Composition
- Depends on: None
- Compatible with: None
- Conflicts with: None
- Related Skills: None
Quick Start
Assumptions
- Production environment with Kubernetes
- PostgreSQL database
- AWS cloud infrastructure
- PagerDuty for alerting
- Slack for team communication
- Git for version control
Compatibility
- Kubernetes: 1.20+
- PostgreSQL: 12+
- AWS CLI: 2.0+
- PagerDuty API: Latest
- Slack API: Latest
Test Scenario Matrix
| Scenario |
Input |
Expected Output |
Verification |
| Service down incident |
Alert notification |
Runbook executed, service restored |
Incident report |
| Production deployment |
New version |
Deployment successful, no issues |
Smoke tests pass |
| High database CPU |
CPU alert |
Query identified and resolved |
CPU returns to normal |
| Database backup |
Scheduled job |
Backup created and uploaded |
S3 verification |
| On-call handoff |
Shift change |
Handoff notes documented |
Next engineer confirms |
Technical Guardrails
Agent Directives & Error Recovery
(ข้อกำหนดสำหรับ AI Agent ในการคิดและแก้ปัญหาเมื่อเกิดข้อผิดพลาด)
- Thinking Process: Analyze root cause before fixing. Do not brute-force.
- Fallback Strategy: Stop after 3 failed test attempts. Output root cause and ask for human intervention/clarification.
- Self-Review: Check against Guardrails & Anti-patterns before finalizing.
- Output Constraints: Output ONLY the modified code block. Do not explain unless asked.
Definition of Done
A runbook is complete when:
Anti-patterns
- Outdated runbooks: Procedures that don't match current system
- Missing steps: Skipping important verification or rollback steps
- Vague instructions: Not specific enough to follow without guessing
- No owner: Unclear who maintains the runbook
- Untested procedures: Runbooks that haven't been validated
- Missing context: Not explaining why steps matter
- No escalation: Unclear when to escalate and to whom
- Copy-paste errors: Commands that don't work as written
Reference Links
Versioning & Changelog
- Version: 1.0.0
- Changelog:
- 2026-02-22: Initial version with complete template structure
1---2name: runbooks-ops-23description: Runbooks are documented procedures for operational tasks, incident response, and troubleshooting. Essential for maintaining reliable systems and enabling team members to handle issues independently.4---5
6# Runbooks Ops
7
8## Skill Profile
9*(Select at least one profile to enable specific modules)*
10- [ ] **DevOps**
11- [x] **Backend**
12- [ ] **Frontend**
13- [ ] **AI-RAG**
14- [ ] **Security Critical**
15
16## Overview
17Runbooks are documented procedures for operational tasks, incident response, and troubleshooting. Essential for maintaining reliable systems and enabling team members to handle issues independently.
18
19## Why This Matters
20- **Faster resolution**: Step-by-step guides reduce MTTR
21- **Consistency**: Same procedure every time
22- **Knowledge sharing**: Reduce bus factor
23- **Onboarding**: New team members can handle ops tasks
24
25## Core Concepts & Rules
26
27### 1. Core Principles
28- Follow established patterns and conventions
29- Maintain consistency across codebase
30- Document decisions and trade-offs
31
32### 2. Implementation Guidelines
33- Start with the simplest viable solution
34- Iterate based on feedback and requirements
35- Test thoroughly before deployment
36
37
38## Inputs / Outputs / Contracts
39* **Inputs**:
40 - <e.g., env vars, request payload, file paths, schema>
41* **Entry Conditions**:
42 - <Pre-requisites: e.g., Repo initialized, DB running, specific branch checked out>
43* **Outputs**:
44 - <e.g., artifacts (PR diff, docs, tests, dashboard JSON)>
45* **Artifacts Required (Deliverables)**:
46 - <e.g., Code Diff, Unit Tests, Migration Script, API Docs>
47* **Acceptance Evidence**:
48 - <e.g., Test Report (screenshot/log), Benchmark Result, Security Scan Report>
49* **Success Criteria**:
50 - <e.g., p95 < 300ms, coverage ≥ 80%>
51
52## Skill Composition
53* **Depends on**: None
54* **Compatible with**: None
55* **Conflicts with**: None
56* **Related Skills**: None
57
58## Quick Start
59#
60
61## Assumptions
62- Production environment with Kubernetes
63- PostgreSQL database
64- AWS cloud infrastructure
65- PagerDuty for alerting
66- Slack for team communication
67- Git for version control
68
69## Compatibility
70- **Kubernetes**: 1.20+
71- **PostgreSQL**: 12+
72- **AWS CLI**: 2.0+
73- **PagerDuty API**: Latest
74- **Slack API**: Latest
75
76## Test Scenario Matrix
77| Scenario | Input | Expected Output | Verification |
78|----------|-------|-----------------|--------------|
79| Service down incident | Alert notification | Runbook executed, service restored | Incident report |
80| Production deployment | New version | Deployment successful, no issues | Smoke tests pass |
81| High database CPU | CPU alert | Query identified and resolved | CPU returns to normal |
82| Database backup | Scheduled job | Backup created and uploaded | S3 verification |
83| On-call handoff | Shift change | Handoff notes documented | Next engineer confirms |
84
85## Technical Guardrails
86#
87
88## Agent Directives & Error Recovery
89*(ข้อกำหนดสำหรับ AI Agent ในการคิดและแก้ปัญหาเมื่อเกิดข้อผิดพลาด)*
90
91- **Thinking Process**: Analyze root cause before fixing. Do not brute-force.
92- **Fallback Strategy**: Stop after 3 failed test attempts. Output root cause and ask for human intervention/clarification.
93- **Self-Review**: Check against Guardrails & Anti-patterns before finalizing.
94- **Output Constraints**: Output ONLY the modified code block. Do not explain unless asked.
95
96
97## Definition of Done
98A runbook is complete when:
99
100- [ ] Follows standard template structure
101- [ ] Has clear overview and when-to-use sections
102- [ ] Lists all prerequisites and required access
103- [ ] Provides step-by-step procedures
104- [ ] Includes copy-paste ready commands
105- [ ] Documents expected outputs
106- [ ] Has verification steps
107- [ ] Includes rollback procedures
108- [ ] Specifies escalation paths
109- [ ] Links to related runbooks
110- [ ] Has owner and last updated date
111- [ ] Has been tested in staging
112
113## Anti-patterns
1141. **Outdated runbooks**: Procedures that don't match current system
1152. **Missing steps**: Skipping important verification or rollback steps
1163. **Vague instructions**: Not specific enough to follow without guessing
1174. **No owner**: Unclear who maintains the runbook
1185. **Untested procedures**: Runbooks that haven't been validated
1196. **Missing context**: Not explaining why steps matter
1207. **No escalation**: Unclear when to escalate and to whom
1218. **Copy-paste errors**: Commands that don't work as written
122
123## Reference Links
124- [Kubernetes Documentation](https://kubernetes.io/docs/)
125- [PostgreSQL Documentation](https://www.postgresql.org/docs/)
126- [AWS Documentation](https://docs.aws.amazon.com/)
127- [PagerDuty Documentation](https://support.pagerduty.com/)
128- [Incident Response Best Practices](https://sre.google/sre-book/postmortem-culture/)
129
130## Versioning & Changelog
131
132* **Version**: 1.0.0
133* **Changelog**:
134 - 2026-02-22: Initial version with complete template structure
135