# Chaos Engineering

> Resilience testing through controlled experiments

- Skill: `ffsshhttiikk/chaos-engineering` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ffsshhttiikk/chaos-engineering`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ffsshhttiikk/chaos-engineering/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: MIT
- Author: ffsshhttiikk (https://skillmd.com/u/ffsshhttiikk)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ffsshhttiikk/chaos-engineering

---


## What I do

- Design and execute chaos experiments to test system resilience
- Create controlled failure scenarios in production-like environments
- Build automated chaos testing pipelines
- Analyze system behavior under failure conditions
- Improve system robustness based on experiment results
- Document failure recovery procedures

## When to use me

- When you want to proactively find system weaknesses
- When validating disaster recovery procedures
- When building confidence in system resilience
- When testing multi-region failover capabilities
- When training incident response teams
- When validating auto-scaling and self-healing behaviors

## Key Concepts

### Chaos Mesh Example

```yaml
apiVersion: chaos-mesh.org/v1alpha1
kind: PodChaos
metadata:
  name: pod-failure
spec:
  action: pod-failure
  mode: one
  duration: "30s"
  selector:
    labelSelectors:
      app: payment-service
```

### LitmusChaos Experiment

```yaml
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: pod-delete-chaos
spec:
  appinfo:
    appns: production
    applabel: "app=payment-service"
  chaosServiceAccount: litmus-admin
  experiments:
    - name: pod-delete
      spec:
        components:
          env:
            - name: TOTAL_CHAOS_DURATION
              value: "30"
            - name: CHAOS_INTERVAL
              value: "10"
```

### Experiment Categories

- **Infrastructure**: Network latency, packet loss, disk failure
- **Compute**: CPU exhaustion, memory pressure, process kill
- **Stateful**: Database failures, cache eviction, storage limits
- **Network**: DNS failures, firewall rules, service isolation
- **Security**: Certificate expiration, authentication failures
- **Scaling**: Node removal, cluster downscaling

### Chaos Engineering Principles

1. **Define steady state**: What does normal behavior look like?
2. **Hypothesize**: What do you expect will happen?
3. **Run experiment**: Introduce real-world failure
4. **Verify**: Did the system behave as expected?
5. **Improve**: Fix发现的 weaknesses
6. **Repeat**: Continuously test and improve

### Observability Integration

```python
# Prometheus alerts during chaos
- alert: HighErrorRateDuringChaos
  expr: |
    sum(rate(http_requests_total{status=~"5.."}[5m])) 
    / sum(rate(http_requests_total[5m])) > 0.05
  for: 2m
  labels:
    chaos_experiment: "active"
```

### Safe Experimentation

- Start in non-production environments
- Define clear abort conditions
- Limit blast radius (one service at a time)
- Have rollback plans ready
- Communicate with stakeholders
- Monitor experiments in real-time
- Document all findings