Automating IOC Enrichment
When to Use
Use this skill when:
- Building a SOAR playbook that automatically enriches SIEM alerts with threat intelligence context before routing to analysts
- Creating a Python pipeline for bulk IOC enrichment from phishing email submissions
- Reducing analyst mean time to triage (MTTT) by pre-populating alert context with VT, Shodan, and MISP data
Do not use this skill for fully automated blocking decisions without human review — enrichment automation should inform decisions, not execute blocks autonomously for high-impact actions.
Prerequisites
- SOAR platform (Cortex XSOAR, Splunk SOAR, Tines, or n8n) or Python 3.9+ environment
- API keys: VirusTotal, AbuseIPDB, Shodan, and at minimum one TIP (MISP or OpenCTI)
- SIEM integration endpoint for alert consumption
- Rate limit budgets documented per API (VT: 4/min free, 500/min enterprise)
Workflow
Step 1: Design Enrichment Pipeline Architecture
Define the enrichment flow for each IOC type:
SIEM Alert → Extract IOCs → Classify Type → Route to enrichment functions
IP Address → AbuseIPDB + Shodan + VirusTotal IP + MISP
Domain → VirusTotal Domain + PassiveTotal + Shodan + MISP
URL → URLScan.io + VirusTotal URL + Google Safe Browse
File Hash → VirusTotal Files + MalwareBazaar + MISP
→ Aggregate results → Calculate confidence score → Update alert → Notify analyst
Step 2: Implement Python Enrichment Functions
import requests
import time
from dataclasses import dataclass, field
from typing import Optional
RATE_LIMIT_DELAY = 0.25 # 4 requests/second for VT free tier
@dataclass
class EnrichmentResult:
ioc_value: str
ioc_type: str
vt_malicious: int = 0
vt_total: int = 0
abuse_confidence: int = 0
shodan_ports: list = field(default_factory=list)
misp_events: list = field(default_factory=list)
confidence_score: int = 0
def enrich_ip(ip: str, vt_key: str, abuse_key: str, shodan_key: str) -> EnrichmentResult:
result = EnrichmentResult(ip, "ip")
# VirusTotal IP lookup
vt_resp = requests.get(
f"https://www.virustotal.com/api/v3/ip_addresses/{ip}",
headers={"x-apikey": vt_key}
)
if vt_resp.status_code == 200:
stats = vt_resp.json()["data"]["attributes"]["last_analysis_stats"]
result.vt_malicious = stats.get("malicious", 0)
result.vt_total = sum(stats.values())
time.sleep(RATE_LIMIT_DELAY)
# AbuseIPDB
abuse_resp = requests.get(
"https://api.abuseipdb.com/api/v2/check",
headers={"Key": abuse_key, "Accept": "application/json"},
params={"ipAddress": ip, "maxAgeInDays": 90}
)
if abuse_resp.status_code == 200:
result.abuse_confidence = abuse_resp.json()["data"]["abuseConfidenceScore"]
# Calculate composite confidence score
result.confidence_score = min(
(result.vt_malicious / max(result.vt_total, 1)) * 60 +
(result.abuse_confidence / 100) * 40, 100
)
return result
def enrich_hash(sha256: str, vt_key: str) -> EnrichmentResult:
result = EnrichmentResult(sha256, "sha256")
vt_resp = requests.get(
f"https://www.virustotal.com/api/v3/files/{sha256}",
headers={"x-apikey": vt_key}
)
if vt_resp.status_code == 200:
stats = vt_resp.json()["data"]["attributes"]["last_analysis_stats"]
result.vt_malicious = stats.get("malicious", 0)
result.vt_total = sum(stats.values())
result.confidence_score = int((result.vt_malicious / max(result.vt_total, 1)) * 100)
return result
Step 3: Build SOAR Playbook (Cortex XSOAR)
In Cortex XSOAR, create an enrichment playbook:
- Trigger: Alert created in SIEM (via webhook or polling)
- Extract IOCs: Use "Extract Indicators" task with regex patterns for IP, domain, URL, hash
- Parallel enrichment: Fan-out to multiple enrichment tasks simultaneously
- VT Enrichment: Call
!vt-file-scan or !vt-ip-scan commands
- AbuseIPDB check: Call
!abuseipdb-check-ip command
- MISP Lookup: Call
!misp-search for cross-referencing
- Score aggregation: Python transform task computing composite score
- Conditional routing: If score ≥70 → High Priority queue; if 40–69 → Medium; <40 → Auto-close with note
- Alert enrichment: Write enrichment results to alert context for analyst view
Step 4: Handle Rate Limiting and Failures
import time
from functools import wraps
def rate_limited(max_per_second):
min_interval = 1.0 / max_per_second
def decorator(func):
last_called = [0.0]
@wraps(func)
def wrapper(*args, **kwargs):
elapsed = time.time() - last_called[0]
wait = min_interval - elapsed
if wait > 0:
time.sleep(wait)
result = func(*args, **kwargs)
last_called[0] = time.time()
return result
return wrapper
return decorator
def retry_on_429(max_retries=3):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(max_retries):
response = func(*args, **kwargs)
if response.status_code == 429:
retry_after = int(response.headers.get("Retry-After", 60))
time.sleep(retry_after)
else:
return response
return wrapper
return decorator
Step 5: Metrics and Tuning
Track pipeline performance weekly:
- Enrichment latency: Target <30 seconds from alert trigger to enriched output
- API success rate: Target >99% (identify rate limit or outage events)
- True positive rate: Track analyst overrides of automated confidence scores
- Cost: Track API call volume against budget (VT Enterprise: $X per 1M lookups)
Key Concepts
| Term |
Definition |
| SOAR |
Security Orchestration, Automation, and Response — platform for automating security workflows and integrating disparate tools |
| Enrichment Playbook |
Automated workflow sequence that adds contextual intelligence to raw security events |
| Rate Limiting |
API provider restrictions on request frequency (e.g., VT free: 4 requests/minute); pipelines must respect these limits |
| Composite Confidence Score |
Single score aggregating signals from multiple enrichment sources using weighted formula |
| Fan-out Pattern |
Parallel execution of multiple enrichment queries simultaneously to minimize total enrichment latency |
Tools & Systems
- Cortex XSOAR (Palo Alto): Enterprise SOAR with 700+ marketplace integrations including VT, MISP, Shodan, and AbuseIPDB
- Splunk SOAR (Phantom): SOAR platform with Python-based playbooks; native Splunk SIEM integration
- Tines: No-code SOAR platform with webhook-driven automation; cost-effective for smaller teams
- TheHive + Cortex: Open-source IR/enrichment platform with observable enrichment via Cortex analyzers
Common Pitfalls
- Blocking on enrichment latency: If enrichment takes >5 minutes, analysts start working unenriched alerts, defeating the purpose. Set timeout limits and provide partial results.
- No caching: Querying the same IOC 50 times generates unnecessary API costs. Cache enrichment results for 24 hours by default.
- Ignoring API failures silently: Failed enrichment calls should be logged and trigger fallback logic, not silently produce empty results that appear as clean IOCs.
- Automating blocks on enrichment score alone: Composite scores contain false positives; require human confirmation for blocking decisions against shared infrastructure.
1---2name: automating-ioc-enrichment3description: >4 Automates the enrichment of raw indicators of compromise with multi-source threat intelligence5 context using SOAR platforms, Python pipelines, or TIP playbooks to reduce analyst triage time6 and standardize enrichment outputs. Use when building automated enrichment workflows integrated7 with SIEM alerts, email submission pipelines, or bulk IOC processing from threat feeds. Activates8 for requests involving SOAR enrichment, Cortex XSOAR, Splunk SOAR, TheHive, Python enrichment9 pipelines, or automated IOC processing.10domain: cybersecurity11subdomain: threat-intelligence12tags: [SOAR, enrichment, IOC, Cortex-XSOAR, Splunk-SOAR, VirusTotal, automation, CTI, NIST-CSF]13version: 1.0.014author: team-cybersecurity15license: MIT16metadata:17 source: https://skillmd.com/skills/njones17/automating-ioc-enrichment18---1920# Automating IOC Enrichment2122## When to Use2324Use this skill when:25- Building a SOAR playbook that automatically enriches SIEM alerts with threat intelligence context before routing to analysts26- Creating a Python pipeline for bulk IOC enrichment from phishing email submissions27- Reducing analyst mean time to triage (MTTT) by pre-populating alert context with VT, Shodan, and MISP data2829**Do not use** this skill for fully automated blocking decisions without human review — enrichment automation should inform decisions, not execute blocks autonomously for high-impact actions.3031## Prerequisites3233- SOAR platform (Cortex XSOAR, Splunk SOAR, Tines, or n8n) or Python 3.9+ environment34- API keys: VirusTotal, AbuseIPDB, Shodan, and at minimum one TIP (MISP or OpenCTI)35- SIEM integration endpoint for alert consumption36- Rate limit budgets documented per API (VT: 4/min free, 500/min enterprise)3738## Workflow3940### Step 1: Design Enrichment Pipeline Architecture4142Define the enrichment flow for each IOC type:43```44SIEM Alert → Extract IOCs → Classify Type → Route to enrichment functions45 IP Address → AbuseIPDB + Shodan + VirusTotal IP + MISP46 Domain → VirusTotal Domain + PassiveTotal + Shodan + MISP47 URL → URLScan.io + VirusTotal URL + Google Safe Browse48 File Hash → VirusTotal Files + MalwareBazaar + MISP49→ Aggregate results → Calculate confidence score → Update alert → Notify analyst50```5152### Step 2: Implement Python Enrichment Functions5354```python55import requests56import time57from dataclasses import dataclass, field58from typing import Optional5960RATE_LIMIT_DELAY = 0.25 # 4 requests/second for VT free tier6162@dataclass63class EnrichmentResult:64 ioc_value: str65 ioc_type: str66 vt_malicious: int = 067 vt_total: int = 068 abuse_confidence: int = 069 shodan_ports: list = field(default_factory=list)70 misp_events: list = field(default_factory=list)71 confidence_score: int = 07273def enrich_ip(ip: str, vt_key: str, abuse_key: str, shodan_key: str) -> EnrichmentResult:74 result = EnrichmentResult(ip, "ip")7576 # VirusTotal IP lookup77 vt_resp = requests.get(78 f"https://www.virustotal.com/api/v3/ip_addresses/{ip}",79 headers={"x-apikey": vt_key}80 )81 if vt_resp.status_code == 200:82 stats = vt_resp.json()["data"]["attributes"]["last_analysis_stats"]83 result.vt_malicious = stats.get("malicious", 0)84 result.vt_total = sum(stats.values())8586 time.sleep(RATE_LIMIT_DELAY)8788 # AbuseIPDB89 abuse_resp = requests.get(90 "https://api.abuseipdb.com/api/v2/check",91 headers={"Key": abuse_key, "Accept": "application/json"},92 params={"ipAddress": ip, "maxAgeInDays": 90}93 )94 if abuse_resp.status_code == 200:95 result.abuse_confidence = abuse_resp.json()["data"]["abuseConfidenceScore"]9697 # Calculate composite confidence score98 result.confidence_score = min(99 (result.vt_malicious / max(result.vt_total, 1)) * 60 +100 (result.abuse_confidence / 100) * 40, 100101 )102103 return result104105def enrich_hash(sha256: str, vt_key: str) -> EnrichmentResult:106 result = EnrichmentResult(sha256, "sha256")107 vt_resp = requests.get(108 f"https://www.virustotal.com/api/v3/files/{sha256}",109 headers={"x-apikey": vt_key}110 )111 if vt_resp.status_code == 200:112 stats = vt_resp.json()["data"]["attributes"]["last_analysis_stats"]113 result.vt_malicious = stats.get("malicious", 0)114 result.vt_total = sum(stats.values())115 result.confidence_score = int((result.vt_malicious / max(result.vt_total, 1)) * 100)116 return result117```118119### Step 3: Build SOAR Playbook (Cortex XSOAR)120121In Cortex XSOAR, create an enrichment playbook:1221. **Trigger**: Alert created in SIEM (via webhook or polling)1232. **Extract IOCs**: Use "Extract Indicators" task with regex patterns for IP, domain, URL, hash1243. **Parallel enrichment**: Fan-out to multiple enrichment tasks simultaneously1254. **VT Enrichment**: Call `!vt-file-scan` or `!vt-ip-scan` commands1265. **AbuseIPDB check**: Call `!abuseipdb-check-ip` command1276. **MISP Lookup**: Call `!misp-search` for cross-referencing1287. **Score aggregation**: Python transform task computing composite score1298. **Conditional routing**: If score ≥70 → High Priority queue; if 40–69 → Medium; <40 → Auto-close with note1309. **Alert enrichment**: Write enrichment results to alert context for analyst view131132### Step 4: Handle Rate Limiting and Failures133134```python135import time136from functools import wraps137138def rate_limited(max_per_second):139 min_interval = 1.0 / max_per_second140 def decorator(func):141 last_called = [0.0]142 @wraps(func)143 def wrapper(*args, **kwargs):144 elapsed = time.time() - last_called[0]145 wait = min_interval - elapsed146 if wait > 0:147 time.sleep(wait)148 result = func(*args, **kwargs)149 last_called[0] = time.time()150 return result151 return wrapper152 return decorator153154def retry_on_429(max_retries=3):155 def decorator(func):156 @wraps(func)157 def wrapper(*args, **kwargs):158 for attempt in range(max_retries):159 response = func(*args, **kwargs)160 if response.status_code == 429:161 retry_after = int(response.headers.get("Retry-After", 60))162 time.sleep(retry_after)163 else:164 return response165 return wrapper166 return decorator167```168169### Step 5: Metrics and Tuning170171Track pipeline performance weekly:172- **Enrichment latency**: Target <30 seconds from alert trigger to enriched output173- **API success rate**: Target >99% (identify rate limit or outage events)174- **True positive rate**: Track analyst overrides of automated confidence scores175- **Cost**: Track API call volume against budget (VT Enterprise: $X per 1M lookups)176177## Key Concepts178179| Term | Definition |180|------|-----------|181| **SOAR** | Security Orchestration, Automation, and Response — platform for automating security workflows and integrating disparate tools |182| **Enrichment Playbook** | Automated workflow sequence that adds contextual intelligence to raw security events |183| **Rate Limiting** | API provider restrictions on request frequency (e.g., VT free: 4 requests/minute); pipelines must respect these limits |184| **Composite Confidence Score** | Single score aggregating signals from multiple enrichment sources using weighted formula |185| **Fan-out Pattern** | Parallel execution of multiple enrichment queries simultaneously to minimize total enrichment latency |186187## Tools & Systems188189- **Cortex XSOAR (Palo Alto)**: Enterprise SOAR with 700+ marketplace integrations including VT, MISP, Shodan, and AbuseIPDB190- **Splunk SOAR (Phantom)**: SOAR platform with Python-based playbooks; native Splunk SIEM integration191- **Tines**: No-code SOAR platform with webhook-driven automation; cost-effective for smaller teams192- **TheHive + Cortex**: Open-source IR/enrichment platform with observable enrichment via Cortex analyzers193194## Common Pitfalls195196- **Blocking on enrichment latency**: If enrichment takes >5 minutes, analysts start working unenriched alerts, defeating the purpose. Set timeout limits and provide partial results.197- **No caching**: Querying the same IOC 50 times generates unnecessary API costs. Cache enrichment results for 24 hours by default.198- **Ignoring API failures silently**: Failed enrichment calls should be logged and trigger fallback logic, not silently produce empty results that appear as clean IOCs.199- **Automating blocks on enrichment score alone**: Composite scores contain false positives; require human confirmation for blocking decisions against shared infrastructure.