Evaluating Threat Intelligence Platforms
When to Use
Use this skill when:
- Conducting a formal RFP or vendor evaluation for a TIP solution
- Assessing whether the current TIP (e.g., MISP) needs to be replaced or augmented as the CTI program scales
- Establishing evaluation criteria aligned to organizational maturity and budget
Do not use this skill for evaluating feed quality independently of the TIP — feed evaluation is a separate workflow focused on data quality rather than platform capabilities.
Detection Gaps & Validation
- PoC coverage gaps: vendor demos use curated feeds. Validate against YOUR top 5 feeds and YOUR SIEM, not the vendor's sample data - STIX 2.1 import that works on clean bundles often fails on real-world malformed objects.
- STIX/TAXII version mismatch: a TIP advertising "STIX support" may only do STIX 2.0 or partial 2.1 (missing
infrastructure/grouping SDOs or sighting SROs). Test round-trip export/import of every SDO/SRO type you actually use.
- Dedup over-merge: aggressive deduplication can collapse distinct indicators (same IP, different context/TLP) and hide detections. Verify TLP and source survive a merge before trusting indicator counts.
- Integration latency: "real-time" SIEM push can lag minutes to hours. Measure actual enriched-IOC-to-SIEM time during the PoC with a planted test indicator, not the datasheet number.
- How to validate: weight the scoring matrix by your real use cases, run the 30-day PoC at production-representative volume (10k+ queries/day), and confirm a known test IOC traverses ingest → dedup → ATT&CK tag → SIEM export intact.
Prerequisites
- Documented CTI program requirements: team size, feed sources, integration targets, use cases
- Budget range and procurement timeline
- Technical staff who will administer the platform (Python/API experience for open-source TIPs)
- List of current and planned integrations (SIEM, SOAR, EDR, firewalls)
Workflow
Step 1: Define Evaluation Criteria
Structure requirements into mandatory (M) and desired (D) categories:
Core TIP Functions:
- M: STIX 2.1 import/export with TAXII 2.1 server
- M: REST API for automated IOC ingestion and export
- M: Indicator deduplication and TTL management
- M: TLP classification enforcement
- D: Built-in MITRE ATT&CK integration and technique tagging
- D: Graph visualization of indicator relationships
- D: Workflow automation for analyst triage
Integrations:
- M: SIEM integration (Splunk, Sentinel, QRadar) via syslog, API, or native connector
- M: EDR integration for IOC push (CrowdStrike, Defender, SentinelOne)
- D: SOAR integration (XSOAR, Splunk SOAR) for playbook triggers
- D: Ticketing system (ServiceNow, Jira) for intelligence task tracking
Operational:
- M: Role-based access control with TLP-aware data segregation
- M: Audit logging for all analyst actions
- D: Multi-tenancy for MSSP use cases
Step 2: Evaluate Major TIP Options
MISP (Open Source):
- Cost: Free (self-hosted infrastructure cost only)
- Strengths: Largest community, 250+ modules, extensive ISAC usage, STIX 2.0 native
- Weaknesses: Requires dedicated admin, limited visualization, UI dated
- Best for: Budget-constrained teams with technical staff; government/ISAC sharing programs
OpenCTI (Open Source):
- Cost: Free (self-hosted); paid SaaS at ~$3,000–$15,000/year
- Strengths: Native STIX 2.1, graph visualization, ATT&CK integration, modern API
- Weaknesses: Resource-intensive deployment (Elasticsearch, MinIO required)
- Best for: Teams wanting open source with modern UX; SOC/CTI integration focus
ThreatConnect (Commercial):
- Cost: $50,000–$500,000/year depending on scale
- Strengths: End-to-end CTI lifecycle, playbook automation, TC Exchange marketplace, analyst workflow
- Weaknesses: High cost; complex implementation; best value at larger scale
- Best for: Mature enterprise CTI programs; MSSPs; red team/blue team integration
Anomali ThreatStream (Commercial):
- Cost: $30,000–$200,000/year
- Strengths: Strong feed aggregation, Splunk-native integration, extensive pre-built connectors
- Weaknesses: Graph visualization weaker than OpenCTI; UI refresh lagging
- Best for: Splunk-heavy environments; teams prioritizing feed volume over analysis workflows
EclecticIQ Platform (Commercial):
- Cost: $40,000–$300,000/year
- Strengths: STIX 2.1 native, collaborative intelligence workbench, strong European customer base
- Weaknesses: Smaller partner ecosystem than ThreatConnect
- Best for: Teams with MITRE ATT&CK-centric workflows; EMEA-focused organizations
Step 3: Conduct Proof of Concept
Request 30-day PoC from finalists. Test:
- Feed onboarding: Can your top 5 feeds be ingested within 4 hours?
- SIEM integration: Can enriched IOCs push to your SIEM in <5 minutes?
- ATT&CK mapping: Can analysts tag indicators with ATT&CK techniques efficiently?
- Report generation: Can the platform produce a tactical IOC bulletin with one click?
- API performance: Can the REST API handle 10,000 indicator queries per day?
Step 4: Score and Select
Use weighted scoring matrix (weight each criterion by organizational priority):
Criterion Weight Vendor A Vendor B
STIX 2.1 compliance 20% 95 85
SIEM integration 25% 90 70
ATT&CK mapping 15% 85 95
Cost (inverse) 20% 60 90
UI/analyst experience 10% 80 75
Vendor support quality 10% 85 80
TOTAL 100% 82.0 81.5
Step 5: Implementation and Onboarding Planning
Plan 90-day implementation:
- Week 1–2: Infrastructure deployment (cloud or on-prem)
- Week 3–4: Feed onboarding and deduplication tuning
- Week 5–6: SIEM/SOAR integration and testing
- Week 7–8: Analyst workflow configuration and training
- Week 9–12: Operational validation and go-live
Key Concepts
| Term |
Definition |
| TIP |
Threat Intelligence Platform — software for collecting, processing, analyzing, and disseminating cyber threat intelligence |
| TAXII Server |
Component of a TIP that serves STIX bundles to consuming systems on request |
| TC Exchange |
ThreatConnect's commercial marketplace for pre-built feed integrations and app connectors |
| Multi-tenancy |
TIP capability to serve multiple organizational units or customers with isolated data environments |
| Deduplication |
Process of identifying and merging duplicate indicators within a TIP to reduce analyst noise |
Tools & Systems
- MISP: Open-source TIP used by 6,000+ organizations; strongest ISAC/government community integration
- OpenCTI: Modern open-source TIP with native STIX 2.1 and graph-based analysis
- ThreatConnect: Enterprise commercial TIP with lifecycle management and SOAR playbook integration
- Anomali ThreatStream: Commercial TIP with strong Splunk ecosystem integration
- EclecticIQ: Commercial TIP with ATT&CK-centric workflow design
Common Pitfalls
- Selecting TIP before defining requirements: Technology selection before use case definition leads to expensive mismatches.
- Underestimating administration burden: MISP and OpenCTI require dedicated admin time (minimum 0.25 FTE); budget accordingly.
- Ignoring data migration costs: Moving historical intelligence from one TIP to another is costly and often impractical for legacy systems.
- Not testing SIEM integration in PoC: TIP value depends heavily on downstream integration quality; always test SIEM/SOAR connectivity during evaluation.
1---2name: evaluating-threat-intelligence-platforms3description: Evaluates and selects Threat Intelligence Platform (TIP) products based on organizational requirements including feed integration capability, STIX/TAXII support, workflow automation, analyst interface, and total cost of ownership. Use when conducting a TIP procurement, migrating between TIP solutions, or assessing whether the current TIP meets program maturity requirements. Activates for requests involving ThreatConnect, MISP, OpenCTI, Anomali, EclecticIQ, or TIP procurement decisions.4license: Apache-2.05---6# Evaluating Threat Intelligence Platforms
7
8## When to Use
9
10Use this skill when:
11- Conducting a formal RFP or vendor evaluation for a TIP solution
12- Assessing whether the current TIP (e.g., MISP) needs to be replaced or augmented as the CTI program scales
13- Establishing evaluation criteria aligned to organizational maturity and budget
14
15**Do not use** this skill for evaluating feed quality independently of the TIP — feed evaluation is a separate workflow focused on data quality rather than platform capabilities.
16
17## Detection Gaps & Validation
18
19- **PoC coverage gaps:** vendor demos use curated feeds. Validate against YOUR top 5 feeds and YOUR SIEM, not the vendor's sample data - STIX 2.1 import that works on clean bundles often fails on real-world malformed objects.
20- **STIX/TAXII version mismatch:** a TIP advertising "STIX support" may only do STIX 2.0 or partial 2.1 (missing `infrastructure`/`grouping` SDOs or `sighting` SROs). Test round-trip export/import of every SDO/SRO type you actually use.
21- **Dedup over-merge:** aggressive deduplication can collapse distinct indicators (same IP, different context/TLP) and hide detections. Verify TLP and source survive a merge before trusting indicator counts.
22- **Integration latency:** "real-time" SIEM push can lag minutes to hours. Measure actual enriched-IOC-to-SIEM time during the PoC with a planted test indicator, not the datasheet number.
23- **How to validate:** weight the scoring matrix by your real use cases, run the 30-day PoC at production-representative volume (10k+ queries/day), and confirm a known test IOC traverses ingest → dedup → ATT&CK tag → SIEM export intact.
24
25## Prerequisites
26
27- Documented CTI program requirements: team size, feed sources, integration targets, use cases
28- Budget range and procurement timeline
29- Technical staff who will administer the platform (Python/API experience for open-source TIPs)
30- List of current and planned integrations (SIEM, SOAR, EDR, firewalls)
31
32## Workflow
33
34### Step 1: Define Evaluation Criteria
35
36Structure requirements into mandatory (M) and desired (D) categories:
37
38**Core TIP Functions**:
39- M: STIX 2.1 import/export with TAXII 2.1 server
40- M: REST API for automated IOC ingestion and export
41- M: Indicator deduplication and TTL management
42- M: TLP classification enforcement
43- D: Built-in MITRE ATT&CK integration and technique tagging
44- D: Graph visualization of indicator relationships
45- D: Workflow automation for analyst triage
46
47**Integrations**:
48- M: SIEM integration (Splunk, Sentinel, QRadar) via syslog, API, or native connector
49- M: EDR integration for IOC push (CrowdStrike, Defender, SentinelOne)
50- D: SOAR integration (XSOAR, Splunk SOAR) for playbook triggers
51- D: Ticketing system (ServiceNow, Jira) for intelligence task tracking
52
53**Operational**:
54- M: Role-based access control with TLP-aware data segregation
55- M: Audit logging for all analyst actions
56- D: Multi-tenancy for MSSP use cases
57
58### Step 2: Evaluate Major TIP Options
59
60**MISP (Open Source)**:
61- Cost: Free (self-hosted infrastructure cost only)
62- Strengths: Largest community, 250+ modules, extensive ISAC usage, STIX 2.0 native
63- Weaknesses: Requires dedicated admin, limited visualization, UI dated
64- Best for: Budget-constrained teams with technical staff; government/ISAC sharing programs
65
66**OpenCTI (Open Source)**:
67- Cost: Free (self-hosted); paid SaaS at ~$3,000–$15,000/year
68- Strengths: Native STIX 2.1, graph visualization, ATT&CK integration, modern API
69- Weaknesses: Resource-intensive deployment (Elasticsearch, MinIO required)
70- Best for: Teams wanting open source with modern UX; SOC/CTI integration focus
71
72**ThreatConnect (Commercial)**:
73- Cost: $50,000–$500,000/year depending on scale
74- Strengths: End-to-end CTI lifecycle, playbook automation, TC Exchange marketplace, analyst workflow
75- Weaknesses: High cost; complex implementation; best value at larger scale
76- Best for: Mature enterprise CTI programs; MSSPs; red team/blue team integration
77
78**Anomali ThreatStream (Commercial)**:
79- Cost: $30,000–$200,000/year
80- Strengths: Strong feed aggregation, Splunk-native integration, extensive pre-built connectors
81- Weaknesses: Graph visualization weaker than OpenCTI; UI refresh lagging
82- Best for: Splunk-heavy environments; teams prioritizing feed volume over analysis workflows
83
84**EclecticIQ Platform (Commercial)**:
85- Cost: $40,000–$300,000/year
86- Strengths: STIX 2.1 native, collaborative intelligence workbench, strong European customer base
87- Weaknesses: Smaller partner ecosystem than ThreatConnect
88- Best for: Teams with MITRE ATT&CK-centric workflows; EMEA-focused organizations
89
90### Step 3: Conduct Proof of Concept
91
92Request 30-day PoC from finalists. Test:
931. Feed onboarding: Can your top 5 feeds be ingested within 4 hours?
942. SIEM integration: Can enriched IOCs push to your SIEM in <5 minutes?
953. ATT&CK mapping: Can analysts tag indicators with ATT&CK techniques efficiently?
964. Report generation: Can the platform produce a tactical IOC bulletin with one click?
975. API performance: Can the REST API handle 10,000 indicator queries per day?
98
99### Step 4: Score and Select
100
101Use weighted scoring matrix (weight each criterion by organizational priority):
102```
103Criterion Weight Vendor A Vendor B
104STIX 2.1 compliance 20% 95 85
105SIEM integration 25% 90 70
106ATT&CK mapping 15% 85 95
107Cost (inverse) 20% 60 90
108UI/analyst experience 10% 80 75
109Vendor support quality 10% 85 80
110TOTAL 100% 82.0 81.5
111```
112
113### Step 5: Implementation and Onboarding Planning
114
115Plan 90-day implementation:
116- Week 1–2: Infrastructure deployment (cloud or on-prem)
117- Week 3–4: Feed onboarding and deduplication tuning
118- Week 5–6: SIEM/SOAR integration and testing
119- Week 7–8: Analyst workflow configuration and training
120- Week 9–12: Operational validation and go-live
121
122## Key Concepts
123
124| Term | Definition |
125|------|-----------|
126| **TIP** | Threat Intelligence Platform — software for collecting, processing, analyzing, and disseminating cyber threat intelligence |
127| **TAXII Server** | Component of a TIP that serves STIX bundles to consuming systems on request |
128| **TC Exchange** | ThreatConnect's commercial marketplace for pre-built feed integrations and app connectors |
129| **Multi-tenancy** | TIP capability to serve multiple organizational units or customers with isolated data environments |
130| **Deduplication** | Process of identifying and merging duplicate indicators within a TIP to reduce analyst noise |
131
132## Tools & Systems
133
134- **MISP**: Open-source TIP used by 6,000+ organizations; strongest ISAC/government community integration
135- **OpenCTI**: Modern open-source TIP with native STIX 2.1 and graph-based analysis
136- **ThreatConnect**: Enterprise commercial TIP with lifecycle management and SOAR playbook integration
137- **Anomali ThreatStream**: Commercial TIP with strong Splunk ecosystem integration
138- **EclecticIQ**: Commercial TIP with ATT&CK-centric workflow design
139
140## Common Pitfalls
141
142- **Selecting TIP before defining requirements**: Technology selection before use case definition leads to expensive mismatches.
143- **Underestimating administration burden**: MISP and OpenCTI require dedicated admin time (minimum 0.25 FTE); budget accordingly.
144- **Ignoring data migration costs**: Moving historical intelligence from one TIP to another is costly and often impractical for legacy systems.
145- **Not testing SIEM integration in PoC**: TIP value depends heavily on downstream integration quality; always test SIEM/SOAR connectivity during evaluation.
gabrielmoreira