Triaging Security Incidents with IR Playbooks
When to Use
- New security alert received from SIEM, EDR, or other detection sources
- SOC analyst needs to determine if an alert is a true positive requiring response
- Incident needs severity classification and team assignment
- Multiple concurrent incidents require prioritization
- Automated triage rules need validation or tuning
Detection Gaps & Validation
- Enrichment context, not just reputation, drives severity: a "clean" VirusTotal/AbuseIPDB result does not downgrade an alert — newly registered C2 and fast-flux infrastructure often have zero detections. Validate against asset criticality (CMDB), data classification, and whether the threat is active vs historical before assigning P1-P4.
- The most-missed triage error is closing the alert in isolation: before calling true/false positive, pivot — same src/dest IP, same user, same hash across the last 30 days. A single brute-force or "quarantined malware" alert is frequently one node of a broader intrusion (lateral movement, persistence) that single-alert triage buries.
- Confirm the alert maps to the right playbook: signature names lie. Decode the payload, resolve the MITRE technique, and verify the trigger conditions actually match before launching a playbook — a mis-categorized incident routes to the wrong team and burns SLA.
- Cross-corroborate the severity inputs: verify asset criticality from CMDB (not the analyst's guess), confirm the account is actually privileged, and validate "active threat" with EDR process state rather than the alert timestamp alone.
- FP tuning: track each detection rule's historical true-positive rate and suppress/auto-close chronic noisemakers (scanner traffic, known admin tooling, sanctioned data flows) so analysts don't fatigue and miss the real P1. Don't auto-close on a low score until enrichment and historical correlation agree.
Prerequisites
- SIEM platform with alert correlation (Splunk, Elastic, QRadar, Sentinel)
- Incident response playbook library (by incident type)
- Severity classification matrix approved by CISO
- On-call rotation and escalation procedures
- Ticketing system for incident tracking (ServiceNow, Jira, TheHive)
- Threat intelligence feeds for IOC enrichment
Workflow
Step 1: Receive and Acknowledge Alert
# Query Splunk for new critical/high severity alerts
index=notable status=new severity IN ("critical","high")
| table _time, rule_name, src, dest, severity, description
| sort -_time
# Query TheHive for new cases
curl -s -H "Authorization: Bearer $THEHIVE_API_KEY" \
"https://thehive.local/api/v1/query?name=list-alerts" \
-H "Content-Type: application/json" \
-d '{"query":[{"_name":"listAlert"},{"_name":"filter","_field":"status","_value":"New"}]}'
# Acknowledge alert in SIEM to prevent duplicate triage
curl -X POST "https://splunk.local:8089/services/notable_update" \
-H "Authorization: Bearer $SPLUNK_TOKEN" \
-d "ruleUIDs=$RULE_UID&status=1&comment=Triage+initiated+by+analyst"
Step 2: Enrich Alert Data
# Enrich source IP with VirusTotal
curl -s "https://www.virustotal.com/api/v3/ip_addresses/$SRC_IP" \
-H "x-apikey: $VT_API_KEY" | jq '.data.attributes.last_analysis_stats'
# Check IP reputation with AbuseIPDB
curl -s "https://api.abuseipdb.com/api/v2/check?ipAddress=$SRC_IP&maxAgeInDays=90" \
-H "Key: $ABUSEIPDB_KEY" -H "Accept: application/json" | jq '.data'
# Enrich file hash with threat intelligence
curl -s "https://www.virustotal.com/api/v3/files/$FILE_HASH" \
-H "x-apikey: $VT_API_KEY" | jq '.data.attributes.last_analysis_stats'
# Query internal asset database for affected systems
curl -s "https://cmdb.local/api/assets?ip=$DEST_IP" \
-H "Authorization: Bearer $CMDB_TOKEN" | jq '.asset_criticality, .owner, .environment'
Step 3: Classify Incident Type
# Map alert to incident category using playbook lookup
# Categories: Malware, Phishing, Unauthorized Access, Data Exfiltration,
# DoS/DDoS, Insider Threat, Ransomware, Account Compromise, Web Attack
# Check if alert matches known playbook trigger conditions
grep -i "$ALERT_SIGNATURE" /opt/ir/playbooks/trigger_conditions.yaml
# Determine incident type from MITRE ATT&CK technique
curl -s "https://attack.mitre.org/api/techniques/$TECHNIQUE_ID" | jq '.name, .tactic'
Step 4: Assign Severity Level
# Severity matrix factors:
# 1. Asset criticality (Critical/High/Medium/Low)
# 2. Data sensitivity (PII/PHI/PCI/Confidential/Public)
# 3. Number of affected systems
# 4. Active vs historical threat
# 5. Confirmed vs suspected compromise
# Automated severity calculation
python3 -c "
severity_score = 0
# Asset criticality: Critical=4, High=3, Medium=2, Low=1
severity_score += 4 # Critical server
# Data sensitivity: PII/PHI=4, PCI=3, Confidential=2, Public=1
severity_score += 3 # PCI data
# Scope: Enterprise=4, Department=3, Single system=2, Single user=1
severity_score += 2 # Single system
# Threat status: Active=4, Recent=3, Historical=2, Potential=1
severity_score += 4 # Active threat
if severity_score >= 12: print('CRITICAL - P1')
elif severity_score >= 9: print('HIGH - P2')
elif severity_score >= 6: print('MEDIUM - P3')
else: print('LOW - P4')
print(f'Score: {severity_score}/16')
"
Step 5: Select and Initiate Playbook
# Load appropriate playbook based on incident type
cat /opt/ir/playbooks/ransomware_playbook.yaml
cat /opt/ir/playbooks/phishing_playbook.yaml
cat /opt/ir/playbooks/unauthorized_access_playbook.yaml
# Create incident ticket in TheHive
curl -X POST "https://thehive.local/api/v1/case" \
-H "Authorization: Bearer $THEHIVE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"title": "IR-2024-XXX: [Incident Type] - [Brief Description]",
"description": "Triage summary and initial findings",
"severity": 3,
"tlp": 2,
"pap": 2,
"tags": ["ransomware", "triage-complete"],
"customFields": {
"playbook": {"string": "ransomware_v2"},
"affected_systems": {"integer": 5}
}
}'
Step 6: Assign Response Team
# Check on-call schedule
curl -s "https://pagerduty.com/api/v2/oncalls?schedule_ids[]=$SCHEDULE_ID" \
-H "Authorization: Token token=$PD_TOKEN" | jq '.oncalls[].user.summary'
# Page incident responders based on severity
# P1/Critical: Page IR lead + senior analysts + CISO
# P2/High: Page IR lead + available analysts
# P3/Medium: Assign to next available analyst
# P4/Low: Queue for business hours processing
curl -X POST "https://events.pagerduty.com/v2/enqueue" \
-H "Content-Type: application/json" \
-d '{
"routing_key": "'$PD_ROUTING_KEY'",
"event_action": "trigger",
"payload": {
"summary": "P1 Security Incident: Ransomware detected on PROD-DB-01",
"severity": "critical",
"source": "SIEM-Splunk",
"custom_details": {"incident_id": "IR-2024-042", "playbook": "ransomware_v2"}
}
}'
Step 7: Document Triage Decision and Hand Off
# Update incident ticket with triage summary
curl -X PATCH "https://thehive.local/api/v1/case/$CASE_ID" \
-H "Authorization: Bearer $THEHIVE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"status": "InProgress",
"customFields": {
"triage_analyst": {"string": "analyst_name"},
"triage_time": {"date": '$(date +%s000)'},
"severity_justification": {"string": "Critical asset + active threat + PCI data"}
}
}'
Key Concepts
| Concept |
Description |
| True Positive |
Alert correctly identifying a real security incident |
| False Positive |
Alert incorrectly flagging benign activity as malicious |
| Severity Classification |
Ranking incident priority based on impact and urgency |
| Playbook Selection |
Choosing the appropriate response procedure based on incident type |
| IOC Enrichment |
Adding context to indicators from threat intelligence sources |
| Escalation Threshold |
Criteria triggering escalation to higher severity or management |
| Triage SLA |
Time target for initial assessment (typically 15-30 min for critical) |
Tools & Systems
| Tool |
Purpose |
| Splunk/Elastic/QRadar |
SIEM alert correlation and querying |
| TheHive/SIRP |
Incident case management and playbook tracking |
| VirusTotal/AbuseIPDB |
IOC reputation and enrichment |
| PagerDuty/OpsGenie |
On-call management and alerting |
| MITRE ATT&CK |
Technique classification and mapping |
| Cortex XSOAR |
SOAR platform for automated triage workflows |
Common Scenarios
- Brute Force Alert: Multiple failed logins from single IP. Enrich IP reputation, check geo-location, verify if account was compromised, assign P3 if unsuccessful.
- Malware Detection on Endpoint: AV/EDR quarantined malware. Verify quarantine success, check for lateral movement, assign P2 if persistence detected.
- Suspicious Outbound Traffic: Large data transfer to unknown external IP. Check if known cloud service, verify data classification, assign P1 if exfiltration confirmed.
- Phishing Email Reported: User reports suspicious email. Extract IOCs, check if others received it, assign P2 if credentials were entered.
- Privilege Escalation: User gained admin rights unexpectedly. Verify if authorized change, check for exploitation, assign P1 if unauthorized.
Output Format
- Triage decision document with severity justification
- Incident ticket with assigned playbook and team
- IOC enrichment summary attached to case
- Escalation notification to appropriate stakeholders
- Initial timeline of events from alert data
1---2name: triaging-security-incident-with-ir-playbook3description: Classify and prioritize security incidents using structured IR playbooks to determine severity, assign response teams, and initiate appropriate response procedures.4license: Apache-2.05---67# Triaging Security Incidents with IR Playbooks89## When to Use10- New security alert received from SIEM, EDR, or other detection sources11- SOC analyst needs to determine if an alert is a true positive requiring response12- Incident needs severity classification and team assignment13- Multiple concurrent incidents require prioritization14- Automated triage rules need validation or tuning1516## Detection Gaps & Validation1718- **Enrichment context, not just reputation, drives severity:** a "clean" VirusTotal/AbuseIPDB result does not downgrade an alert — newly registered C2 and fast-flux infrastructure often have zero detections. Validate against asset criticality (CMDB), data classification, and whether the threat is *active* vs historical before assigning P1-P4.19- **The most-missed triage error is closing the alert in isolation:** before calling true/false positive, pivot — same src/dest IP, same user, same hash across the last 30 days. A single brute-force or "quarantined malware" alert is frequently one node of a broader intrusion (lateral movement, persistence) that single-alert triage buries.20- **Confirm the alert maps to the right playbook:** signature names lie. Decode the payload, resolve the MITRE technique, and verify the trigger conditions actually match before launching a playbook — a mis-categorized incident routes to the wrong team and burns SLA.21- **Cross-corroborate the severity inputs:** verify asset criticality from CMDB (not the analyst's guess), confirm the account is actually privileged, and validate "active threat" with EDR process state rather than the alert timestamp alone.22- **FP tuning:** track each detection rule's historical true-positive rate and suppress/auto-close chronic noisemakers (scanner traffic, known admin tooling, sanctioned data flows) so analysts don't fatigue and miss the real P1. Don't auto-close on a low score until enrichment and historical correlation agree.2324## Prerequisites25- SIEM platform with alert correlation (Splunk, Elastic, QRadar, Sentinel)26- Incident response playbook library (by incident type)27- Severity classification matrix approved by CISO28- On-call rotation and escalation procedures29- Ticketing system for incident tracking (ServiceNow, Jira, TheHive)30- Threat intelligence feeds for IOC enrichment3132## Workflow3334### Step 1: Receive and Acknowledge Alert35```bash36# Query Splunk for new critical/high severity alerts37index=notable status=new severity IN ("critical","high")38| table _time, rule_name, src, dest, severity, description39| sort -_time4041# Query TheHive for new cases42curl -s -H "Authorization: Bearer $THEHIVE_API_KEY" \43 "https://thehive.local/api/v1/query?name=list-alerts" \44 -H "Content-Type: application/json" \45 -d '{"query":[{"_name":"listAlert"},{"_name":"filter","_field":"status","_value":"New"}]}'4647# Acknowledge alert in SIEM to prevent duplicate triage48curl -X POST "https://splunk.local:8089/services/notable_update" \49 -H "Authorization: Bearer $SPLUNK_TOKEN" \50 -d "ruleUIDs=$RULE_UID&status=1&comment=Triage+initiated+by+analyst"51```5253### Step 2: Enrich Alert Data54```bash55# Enrich source IP with VirusTotal56curl -s "https://www.virustotal.com/api/v3/ip_addresses/$SRC_IP" \57 -H "x-apikey: $VT_API_KEY" | jq '.data.attributes.last_analysis_stats'5859# Check IP reputation with AbuseIPDB60curl -s "https://api.abuseipdb.com/api/v2/check?ipAddress=$SRC_IP&maxAgeInDays=90" \61 -H "Key: $ABUSEIPDB_KEY" -H "Accept: application/json" | jq '.data'6263# Enrich file hash with threat intelligence64curl -s "https://www.virustotal.com/api/v3/files/$FILE_HASH" \65 -H "x-apikey: $VT_API_KEY" | jq '.data.attributes.last_analysis_stats'6667# Query internal asset database for affected systems68curl -s "https://cmdb.local/api/assets?ip=$DEST_IP" \69 -H "Authorization: Bearer $CMDB_TOKEN" | jq '.asset_criticality, .owner, .environment'70```7172### Step 3: Classify Incident Type73```bash74# Map alert to incident category using playbook lookup75# Categories: Malware, Phishing, Unauthorized Access, Data Exfiltration,76# DoS/DDoS, Insider Threat, Ransomware, Account Compromise, Web Attack7778# Check if alert matches known playbook trigger conditions79grep -i "$ALERT_SIGNATURE" /opt/ir/playbooks/trigger_conditions.yaml8081# Determine incident type from MITRE ATT&CK technique82curl -s "https://attack.mitre.org/api/techniques/$TECHNIQUE_ID" | jq '.name, .tactic'83```8485### Step 4: Assign Severity Level86```bash87# Severity matrix factors:88# 1. Asset criticality (Critical/High/Medium/Low)89# 2. Data sensitivity (PII/PHI/PCI/Confidential/Public)90# 3. Number of affected systems91# 4. Active vs historical threat92# 5. Confirmed vs suspected compromise9394# Automated severity calculation95python3 -c "96severity_score = 097# Asset criticality: Critical=4, High=3, Medium=2, Low=198severity_score += 4 # Critical server99# Data sensitivity: PII/PHI=4, PCI=3, Confidential=2, Public=1100severity_score += 3 # PCI data101# Scope: Enterprise=4, Department=3, Single system=2, Single user=1102severity_score += 2 # Single system103# Threat status: Active=4, Recent=3, Historical=2, Potential=1104severity_score += 4 # Active threat105106if severity_score >= 12: print('CRITICAL - P1')107elif severity_score >= 9: print('HIGH - P2')108elif severity_score >= 6: print('MEDIUM - P3')109else: print('LOW - P4')110print(f'Score: {severity_score}/16')111"112```113114### Step 5: Select and Initiate Playbook115```bash116# Load appropriate playbook based on incident type117cat /opt/ir/playbooks/ransomware_playbook.yaml118cat /opt/ir/playbooks/phishing_playbook.yaml119cat /opt/ir/playbooks/unauthorized_access_playbook.yaml120121# Create incident ticket in TheHive122curl -X POST "https://thehive.local/api/v1/case" \123 -H "Authorization: Bearer $THEHIVE_API_KEY" \124 -H "Content-Type: application/json" \125 -d '{126 "title": "IR-2024-XXX: [Incident Type] - [Brief Description]",127 "description": "Triage summary and initial findings",128 "severity": 3,129 "tlp": 2,130 "pap": 2,131 "tags": ["ransomware", "triage-complete"],132 "customFields": {133 "playbook": {"string": "ransomware_v2"},134 "affected_systems": {"integer": 5}135 }136 }'137```138139### Step 6: Assign Response Team140```bash141# Check on-call schedule142curl -s "https://pagerduty.com/api/v2/oncalls?schedule_ids[]=$SCHEDULE_ID" \143 -H "Authorization: Token token=$PD_TOKEN" | jq '.oncalls[].user.summary'144145# Page incident responders based on severity146# P1/Critical: Page IR lead + senior analysts + CISO147# P2/High: Page IR lead + available analysts148# P3/Medium: Assign to next available analyst149# P4/Low: Queue for business hours processing150151curl -X POST "https://events.pagerduty.com/v2/enqueue" \152 -H "Content-Type: application/json" \153 -d '{154 "routing_key": "'$PD_ROUTING_KEY'",155 "event_action": "trigger",156 "payload": {157 "summary": "P1 Security Incident: Ransomware detected on PROD-DB-01",158 "severity": "critical",159 "source": "SIEM-Splunk",160 "custom_details": {"incident_id": "IR-2024-042", "playbook": "ransomware_v2"}161 }162 }'163```164165### Step 7: Document Triage Decision and Hand Off166```bash167# Update incident ticket with triage summary168curl -X PATCH "https://thehive.local/api/v1/case/$CASE_ID" \169 -H "Authorization: Bearer $THEHIVE_API_KEY" \170 -H "Content-Type: application/json" \171 -d '{172 "status": "InProgress",173 "customFields": {174 "triage_analyst": {"string": "analyst_name"},175 "triage_time": {"date": '$(date +%s000)'},176 "severity_justification": {"string": "Critical asset + active threat + PCI data"}177 }178 }'179```180181## Key Concepts182183| Concept | Description |184|---------|-------------|185| True Positive | Alert correctly identifying a real security incident |186| False Positive | Alert incorrectly flagging benign activity as malicious |187| Severity Classification | Ranking incident priority based on impact and urgency |188| Playbook Selection | Choosing the appropriate response procedure based on incident type |189| IOC Enrichment | Adding context to indicators from threat intelligence sources |190| Escalation Threshold | Criteria triggering escalation to higher severity or management |191| Triage SLA | Time target for initial assessment (typically 15-30 min for critical) |192193## Tools & Systems194195| Tool | Purpose |196|------|---------|197| Splunk/Elastic/QRadar | SIEM alert correlation and querying |198| TheHive/SIRP | Incident case management and playbook tracking |199| VirusTotal/AbuseIPDB | IOC reputation and enrichment |200| PagerDuty/OpsGenie | On-call management and alerting |201| MITRE ATT&CK | Technique classification and mapping |202| Cortex XSOAR | SOAR platform for automated triage workflows |203204## Common Scenarios2052061. **Brute Force Alert**: Multiple failed logins from single IP. Enrich IP reputation, check geo-location, verify if account was compromised, assign P3 if unsuccessful.2072. **Malware Detection on Endpoint**: AV/EDR quarantined malware. Verify quarantine success, check for lateral movement, assign P2 if persistence detected.2083. **Suspicious Outbound Traffic**: Large data transfer to unknown external IP. Check if known cloud service, verify data classification, assign P1 if exfiltration confirmed.2094. **Phishing Email Reported**: User reports suspicious email. Extract IOCs, check if others received it, assign P2 if credentials were entered.2105. **Privilege Escalation**: User gained admin rights unexpectedly. Verify if authorized change, check for exploitation, assign P1 if unauthorized.211212## Output Format213- Triage decision document with severity justification214- Incident ticket with assigned playbook and team215- IOC enrichment summary attached to case216- Escalation notification to appropriate stakeholders217- Initial timeline of events from alert data