Performing Dark Web Monitoring For Threats

Performing Dark Web Monitoring for Threats

Overview

Dark web monitoring involves systematically scanning Tor hidden services, underground forums, paste sites, and dark web marketplaces to identify threats targeting an organization, including leaked credentials, data breaches, threat actor discussions, vulnerability exploitation tools, and planned attacks. This skill covers setting up monitoring infrastructure, using Tor-based collection tools, implementing automated alerting for brand mentions and credential leaks, and analyzing dark web intelligence for actionable threat indicators.

When to Use

Trigger phrases:

• "performing dark web monitoring for threats"

• "Dark web monitoring involves systematically scanning Tor hidden services, underg"

• When conducting security assessments that involve performing dark web monitoring for threats

• When following incident response procedures for related security events

• When performing scheduled security testing or auditing activities

• When validating security controls through hands-on testing

Prerequisites

• Tor Browser and Tor proxy (SOCKS5 on port 9050)
• Python 3.9+ with requests, stem, beautifulsoup4, stix2 libraries
• Understanding of Tor hidden service architecture (.onion domains)
• API access to dark web monitoring services (Flare, SpyCloud, DarkOwl, Intel 471)
• Awareness of legal and ethical boundaries for dark web research
• Isolated VM for dark web browsing (no personal or corporate identity leakage)

Key Concepts

This section covers key concepts for performing dark web monitoring for threats.

• Ensure all prerequisites are met before proceeding
• Follow the documented workflow steps in sequence
• Record results and any anomalies encountered during this phase

Dark Web Intelligence Sources

• Underground Forums: Hacking forums where threat actors discuss TTPs, sell exploits, and share tools
• Paste Sites: Platforms for sharing stolen data, credentials, and code snippets
• Marketplaces: Dark web markets selling stolen data, RaaS, exploit kits, and access
• Telegram/Discord: Alternative communication channels for cybercriminal groups
• Ransomware Leak Sites: Blogs where ransomware groups post stolen data from victims

Collection Methods

• Automated Crawling: Tor-based web crawlers scanning hidden services
• API-Based Monitoring: Commercial dark web monitoring APIs (Flare, DarkOwl, Intel 471)
• Manual HUMINT: Analyst-driven research on specific forums and marketplaces
• Credential Monitoring: Breach databases and paste site monitoring for leaked credentials

OPSEC for Dark Web Research

• Use dedicated VMs with no personal data
• Route all traffic through Tor (Whonix or Tails recommended)
• Never use personal accounts or identifiable information
• Use separate email addresses and personas for forum registration
• Disable JavaScript in Tor Browser for enhanced security
• Never download or execute files from dark web sources on production systems

Workflow

1. Scope the task — define objectives, boundaries, and success criteria
2. Gather information — collect all necessary data and context before proceeding
3. Execute the core workflow — follow the domain-specific steps methodically
4. Validate results — verify outputs against expected outcomes or baselines
5. Document findings — record results, anomalies, and recommendations

Step 1: Set Up Tor-Based HTTP Client

import requests
from requests.adapters import HTTPAdapter

def create_tor_session():
    """Create a requests session routed through Tor SOCKS5 proxy."""
    session = requests.Session()
    session.proxies = {
        "http": "socks5h://127.0.0.1:9050",
        "https": "socks5h://127.0.0.1:9050",
    }
    session.headers.update({
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; rv:109.0) Gecko/20100101 Firefox/115.0",
    })
    return session


def verify_tor_connection(session):
    """Verify that traffic is routed through Tor."""
    try:
        resp = session.get("https://check.torproject.org/api/ip", timeout=30)
        data = resp.json()
        return {
            "is_tor": data.get("IsTor", False),
            "ip": data.get("IP", ""),
        }
    except Exception as e:
        return {"error": str(e)}

Step 2: Monitor Paste Sites for Credential Leaks

import re
from datetime import datetime

def monitor_paste_sites(session, organization_domains):
    """Monitor paste sites for leaked credentials matching organization domains."""
    findings = []

    # Check Have I Been Pwned API (clearnet)
    for domain in organization_domains:
        try:
            resp = requests.get(
                f"https://haveibeenpwned.com/api/v3/breaches",
                headers={"hibp-api-key": "YOUR_HIBP_KEY"},
                timeout=30,
            )
            if resp.status_code == 200:
                breaches = resp.json()
                for breach in breaches:
                    if domain.lower() in breach.get("Domain", "").lower():
                        findings.append({
                            "source": "HIBP",
                            "breach_name": breach["Name"],
                            "breach_date": breach.get("BreachDate"),
                            "data_classes": breach.get("DataClasses", []),
                            "pwn_count": breach.get("PwnCount", 0),
                            "domain": domain,
                        })
        except Exception as e:
            print(f"[-] HIBP error for {domain}: {e}")

    return findings


def search_for_keywords(session, keywords, onion_paste_urls):
    """Search dark web paste sites for specific keywords."""
    results = []

    for paste_url in onion_paste_urls:
        try:
            resp = session.get(paste_url, timeout=60)
            if resp.status_code == 200:
                content = resp.text.lower()
                for keyword in keywords:
                    if keyword.lower() in content:
                        results.append({
                            "url": paste_url,
                            "keyword": keyword,
                            "timestamp": datetime.utcnow().isoformat(),
                            "snippet": extract_context(content, keyword.lower()),
                        })
        except Exception as e:
            print(f"[-] Error fetching {paste_url}: {e}")

    return results


def extract_context(text, keyword, context_chars=200):
    """Extract text context around a keyword match."""
    idx = text.find(keyword)
    if idx == -1:
        return ""
    start = max(0, idx - context_chars)
    end = min(len(text), idx + len(keyword) + context_chars)
    return text[start:end]

Step 3: Monitor Ransomware Leak Sites

def check_ransomware_leak_sites(session, organization_name):
    """Check known ransomware group leak sites for organization mentions."""
    # Use Ransomwatch API (clearnet aggregator of ransomware leak sites)
    try:
        resp = requests.get(
            "https://raw.githubusercontent.com/joshhighet/ransomwatch/main/posts.json",
            timeout=30,
        )
        if resp.status_code == 200:
            posts = resp.json()
            matches = []
            for post in posts:
                post_title = post.get("post_title", "").lower()
                if organization_name.lower() in post_title:
                    matches.append({
                        "group": post.get("group_name", ""),
                        "title": post.get("post_title", ""),
                        "discovered": post.get("discovered", ""),
                        "url": post.get("post_url", ""),
                    })
            return matches
    except Exception as e:
        print(f"[-] Ransomwatch error: {e}")
    return []

Step 4: Generate Dark Web Intelligence Report

def generate_dark_web_report(findings, organization):
    """Generate structured dark web intelligence report."""
    report = {
        "organization": organization,
        "report_date": datetime.utcnow().isoformat(),
        "executive_summary": "",
        "credential_leaks": [],
        "ransomware_mentions": [],
        "dark_web_mentions": [],
        "recommendations": [],
    }

    for finding in findings:
        if finding.get("source") == "HIBP":
            report["credential_leaks"].append(finding)
        elif finding.get("group"):
            report["ransomware_mentions"].append(finding)
        else:
            report["dark_web_mentions"].append(finding)

    # Generate executive summary
    cred_count = len(report["credential_leaks"])
    ransom_count = len(report["ransomware_mentions"])
    report["executive_summary"] = (
        f"Monitoring identified {cred_count} credential leak sources "
        f"and {ransom_count} ransomware group mentions for {organization}."
    )

    if ransom_count > 0:
        report["recommendations"].append(
            "CRITICAL: Organization mentioned on ransomware leak site. "
            "Initiate incident response immediately."
        )
    if cred_count > 0:
        report["recommendations"].append(
            "HIGH: Leaked credentials detected. Force password resets for "
            "affected accounts and enable MFA."
        )

    return report

Validation Criteria

• Tor connection established and verified via check.torproject.org
• Credential leak monitoring returns results from HIBP and paste sites
• Ransomware leak site monitoring identifies relevant mentions
• Dark web intelligence report generated with actionable recommendations
• All monitoring performed within legal and ethical boundaries
• OPSEC maintained: no personal or corporate identity exposure

When NOT to Use

• You don't have explicit written authorization to test
• Task is about defense/detection, not offense (use detection skills)
• You need to implement security controls (use implementing-* skills)
• Task requires compliance auditing (use auditing-* skills)
• You're investigating an incident (use incident response skills)
• Target is out of scope for your engagement
• Task is about vulnerability scanning only (use scanning tools)

Red Flags

• Performing actions without explicit written authorization from the asset owner
• Testing against production systems without a defined scope and rules of engagement
• Testing without rate limiting, potentially causing service degradation
• Storing sensitive test data (credentials, tokens) in plain text logs
• Using automated scanners blindly without reviewing results for false positives

Verification

• All steps executed successfully against a test environment before production use
• Output documented with screenshots or logs demonstrating expected behavior
• Vulnerabilities reproduced with proof-of-concept and impact analysis
• False positives filtered out through manual verification
• Fix recommendations include code-level remediation guidance

References

• Tor Project
• Have I Been Pwned API
• Ransomwatch
• DarkOwl
• Intel 471
• Flare Systems

Process

1. Analyze the task requirements
2. Apply domain expertise
3. Verify output quality

Anti-Rationalization Table