# Vibeinfra Incident Solver

> Systematic 7-step production SRE incident investigation and root cause mitigation for VibeInfra flight simulators (Linux kernel, Docker, Kubernetes, Nginx, PostgreSQL, Kafka).

- Skill: `vibeinfraid/vibeinfra-incident-solver` (Agent Skill)
- Install (CLI): `npx skillmds@latest add vibeinfraid/vibeinfra-incident-solver`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vibeinfraid/vibeinfra-incident-solver/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: vibeInfraid (https://skillmd.com/u/vibeinfraid)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/vibeinfraid/vibeinfra-incident-solver

---


# VibeInfra Incident Solver Skill

Use this skill when diagnosing, mitigating, and verifying hands-on production incidents in VibeInfra's flight simulators.

## The Canonical 7-Step Production Incident Lifecycle

Every intermediate, advanced, and expert production incident follows this strict lifecycle:

1. **Traffic Shedding & Triage**:
   - Enable gateway rate-limiting or circuit breaker to prevent client retry storms.
   - Inspect edge proxy metrics (`P99 latency`, `5xx error rate`).

2. **Blast Radius Isolation**:
   - Cordon degraded Kubernetes nodes / isolate poisoned backend containers.
   - Prevent cascading failures to upstream dependencies.

3. **Data Tier & Pool Unclog**:
   - Check PostgreSQL `pg_stat_activity`, active locks, and unindexed sequential scans.
   - Terminate hanging idle-in-transaction connections and increase connection pool headroom.

4. **Root Cause Patch**:
   - Apply targeted configuration fix (Nginx `proxy_pass`, `keepalive_timeout`, Kubernetes `readinessProbe`, AWS IAM ARN policy, or sysctl `nf_conntrack_max`).

5. **Queue Drain & State Reconciliation**:
   - Purge Dead Letter Queue (DLQ) if corrupted, flush poisoned cache keys in Redis.

6. **Controlled Progressive Scale-Up**:
   - Gradually increase replica count (1 -> 3 -> 5) to prevent thundering herd on cold databases.

7. **Sustained Traffic Gate & AAR**:
   - Maintain $\ge 30\text{s}$ continuous loadgen verification with $0\%$ error rate.
   - Review diagnostic findings in After-Action Review.