micsapp Speak Incident Runbook
Overview
Rapid incident response procedures for Speak language learning-related outages.
Prerequisites
- Access to Speak dashboard and status page
- kubectl access to production cluster
- Prometheus/Grafana access
- Communication channels (Slack, PagerDuty)
Instructions
- Severity Levels
- Quick Triage
- Decision Tree
- Immediate Actions by Error Type
- Communication Templates
- Fallback Modes
- Post-Incident
For full implementation details, load: Read(${CLAUDE_SKILL_DIR}/references/implementation-guide.md)
Output
- Issue identified and categorized
- Mitigation applied
- Stakeholders notified
- Evidence collected for postmortem
- Fallback modes enabled if needed
Error Handling
| Issue | Cause | Solution |
|---|---|---|
| Can't reach status page | Network issue | Use mobile or VPN |
| kubectl fails | Auth expired | Re-authenticate |
| Metrics unavailable | Prometheus down | Check backup metrics |
| Fallback not working | Cache empty | Pre-warm cache |
Examples
One-Line Health Check
set -euo pipefail
curl -sf https://api.yourapp.com/health | jq '.services.speak.status' || echo "UNHEALTHY"
Quick Fallback Toggle
set -euo pipefail
# Enable fallback
kubectl set env deployment/speak-integration SPEAK_FALLBACK_MODE=true
# Disable fallback (restore normal)
kubectl set env deployment/speak-integration SPEAK_FALLBACK_MODE-
Resources
Next Steps
For data handling, see speak-data-handling.