Performing Osint With Spiderfoot

TL;DR

• 目的：Automate OSINT collection with the SpiderFoot REST API and CLI (sf.py/spiderfoot-cli) across 200+ modules, selecting scan modes (footprint…
• 适用：目标资产侦察/信息收集
• 输入：目标根域名 + 扫描模式（footprint / investigate / passive）
• 输出：资产清单 + 子域列表 + 暴露面报告
• 红线：仅限授权范围内；扫描限速 -c 10 -rl 10；所有动作记 oplog
• 关联：上游：003-src-session-start → 下游：009-conducting-external-reconnaissance-with-osint, 012-performing-ip-reputation-analysis-with-shodan, 011-offensive-osint

Performing OSINT with SpiderFoot

Quick Start

# SpiderFoot 启动扫描
docker run -d --name spiderfoot -p 5001:5001 spiderfoot/spiderfoot
# 浏览器打开 http://localhost:5001 启动扫描

Overview

SpiderFoot is an open-source OSINT automation tool with 200+ modules that integrates with data sources for threat intelligence and attack surface mapping. This skill uses the SpiderFoot REST API and CLI (sf.py/spiderfoot-cli) to create and manage scans, select modules by use case (footprint, investigate, passive), parse structured results for domains, IPs, email addresses, leaked credentials, and DNS records, and generate target intelligence profiles.

When to Use

• When conducting security assessments that involve performing osint with spiderfoot
• When following incident response procedures for related security events
• When performing scheduled security testing or auditing activities
• When validating security controls through hands-on testing

Prerequisites

• SpiderFoot 4.0+ installed or SpiderFoot HX cloud account
• Python 3.8+ with requests library
• SpiderFoot server running on default port 5001
• Optional: API keys for VirusTotal, Shodan, HaveIBeenPwned modules

Workflow

1. Connect to SpiderFoot REST API or use CLI interface
2. Create a new scan with target specification (domain, IP, email, name)
3. Select scan modules by use case (all, footprint, investigate, passive)
4. Monitor scan progress via API polling
5. Retrieve and parse scan results by data element type
6. Extract key findings: subdomains, IPs, emails, leaked credentials
7. Generate structured OSINT intelligence report

Output Format

JSON report containing OSINT findings organized by data type (domains, IPs, emails, credentials, DNS records), module source attribution, and target profile summary with risk indicators.

Tools & Systems

• subfinder — Subdomain enumeration (passive + active)
• amass — Deep subdomain recon with graph database
• shodan — Internet-wide device search
• censys — Certificate transparency + scan data
• theHarvester — Email/subdomain/host harvesting
• SpiderFoot — Automated OSINT (200+ modules)

All tools run locally with -rate-limit 20 to avoid OPSEC issues.

Workflow

1. Seed input — Root domain, company name, or ASN
2. Passive collection — crt.sh, shodan, censys, theHarvester (no touch)
3. Active enumeration — subfinder with public resolvers
4. Validation — Resolve and probe live hosts
5. Fingerprint — Identify technology stack
6. Reporting — Aggregate into recon report

Advanced Techniques

Multi-Source Correlation

Combine passive sources (crt.sh, shodan) with active enumeration results to identify shadow IT, forgotten subdomains, and orphan DNS records.

Timeline Analysis

Use whois history and Certificate Transparency logs to reconstruct the target's infrastructure evolution.