Clay Incident Runbook
Overview
Rapid incident response procedures for Clay-related outages.
Prerequisites
- Access to Clay dashboard and status page
- kubectl access to production cluster
- Prometheus/Grafana access
- Communication channels (Slack, PagerDuty)
Severity Levels
| Level |
Definition |
Response Time |
Examples |
| P1 |
Complete outage |
< 15 min |
Clay API unreachable |
| P2 |
Degraded service |
< 1 hour |
High latency, partial failures |
| P3 |
Minor impact |
< 4 hours |
Webhook delays, non-critical errors |
| P4 |
No user impact |
Next business day |
Monitoring gaps |
Instructions
Step 1: Quick Triage
Check Clay status page, your integration health endpoint, error rate metrics, and recent pod logs.
Step 2: Follow Decision Tree
If Clay API returns errors and status.clay.com shows an incident, wait and enable fallback. If no Clay incident, check your credentials and config. If no API errors but your service is unhealthy, investigate infrastructure.
Step 3: Execute Immediate Actions
- 401/403: Verify API key in secrets, update if rotated, restart pods
- 429: Check rate limit headers, enable request queuing
- 500/503: Enable graceful degradation, monitor Clay status
Step 4: Communicate Status
Post to internal Slack with severity, impact, current action, and next update time. Update external status page with user-facing impact description.
For complete triage scripts, remediation commands, communication templates, and postmortem template, load the reference guide:
Read(${CLAUDE_SKILL_DIR}/references/implementation-guide.md)
Output
- Issue identified and categorized
- Remediation applied
- Stakeholders notified
- Evidence collected for postmortem
Error Handling
| Issue |
Cause |
Solution |
| Can't reach status page |
Network issue |
Use mobile or VPN |
| kubectl fails |
Auth expired |
Re-authenticate |
| Metrics unavailable |
Prometheus down |
Check backup metrics |
| Secret rotation fails |
Permission denied |
Escalate to admin |
Resources
Next Steps
For data handling, see clay-data-handling.
Examples
Basic usage: Apply clay incident runbook to a standard project setup with default configuration options.
Advanced scenario: Customize clay incident runbook for production environments with multiple constraints and team-specific requirements.
1---2name: clay-incident-runbook3description: |4 Execute Clay incident response procedures with triage, mitigation, and postmortem.5 Use when responding to Clay-related outages, investigating errors,6 or running post-incident reviews for Clay integration failures.7 Trigger with phrases like "clay incident", "clay outage",8 "clay down", "clay on-call", "clay emergency", "clay broken".9allowed-tools: Read, Grep, Bash(kubectl:*), Bash(curl:*)10version: 1.0.011license: MIT12author: Jeremy Longshore <jeremy@intentsolutions.io>13compatible-with: claude-code, codex, openclaw14---1516# Clay Incident Runbook1718## Overview1920Rapid incident response procedures for Clay-related outages.2122## Prerequisites2324- Access to Clay dashboard and status page25- kubectl access to production cluster26- Prometheus/Grafana access27- Communication channels (Slack, PagerDuty)2829## Severity Levels3031| Level | Definition | Response Time | Examples |32|-------|------------|---------------|----------|33| P1 | Complete outage | < 15 min | Clay API unreachable |34| P2 | Degraded service | < 1 hour | High latency, partial failures |35| P3 | Minor impact | < 4 hours | Webhook delays, non-critical errors |36| P4 | No user impact | Next business day | Monitoring gaps |3738## Instructions3940### Step 1: Quick Triage4142Check Clay status page, your integration health endpoint, error rate metrics, and recent pod logs.4344### Step 2: Follow Decision Tree4546If Clay API returns errors and status.clay.com shows an incident, wait and enable fallback. If no Clay incident, check your credentials and config. If no API errors but your service is unhealthy, investigate infrastructure.4748### Step 3: Execute Immediate Actions4950- **401/403**: Verify API key in secrets, update if rotated, restart pods51- **429**: Check rate limit headers, enable request queuing52- **500/503**: Enable graceful degradation, monitor Clay status5354### Step 4: Communicate Status5556Post to internal Slack with severity, impact, current action, and next update time. Update external status page with user-facing impact description.5758For complete triage scripts, remediation commands, communication templates, and postmortem template, load the reference guide:59`Read(${CLAUDE_SKILL_DIR}/references/implementation-guide.md)`6061## Output6263- Issue identified and categorized64- Remediation applied65- Stakeholders notified66- Evidence collected for postmortem6768## Error Handling6970| Issue | Cause | Solution |71|-------|-------|----------|72| Can't reach status page | Network issue | Use mobile or VPN |73| kubectl fails | Auth expired | Re-authenticate |74| Metrics unavailable | Prometheus down | Check backup metrics |75| Secret rotation fails | Permission denied | Escalate to admin |7677## Resources7879- [Clay Status Page](https://status.clay.com)80- [Clay Support](https://support.clay.com)8182## Next Steps8384For data handling, see `clay-data-handling`.8586## Examples8788**Basic usage**: Apply clay incident runbook to a standard project setup with default configuration options.8990**Advanced scenario**: Customize clay incident runbook for production environments with multiple constraints and team-specific requirements.
micsapp