Performing Open Source Intelligence Gathering

Performing Open Source Intelligence Gathering

Legal Notice: This skill is for authorized security testing and educational purposes only. Unauthorized use against systems you do not own or have written permission to test is illegal and may violate computer fraud laws.

Overview

Open Source Intelligence (OSINT) gathering is the first active phase of a red team engagement, where operators collect publicly available information about the target organization to identify attack surfaces, potential targets for social engineering, technology stacks, and credential exposures. Effective OSINT directly shapes initial access strategies and reduces operational risk.

When to Use

• When conducting security assessments that involve performing open source intelligence gathering
• When following incident response procedures for related security events
• When performing scheduled security testing or auditing activities
• When validating security controls through hands-on testing

Most Often Missed & How to Confirm

• Breach/credential dumps: check HaveIBeenPwned, Dehashed, and public combolists for @target.com creds — often the fastest path to initial access and the most-skipped source.
• Code & secret leakage: GitHub dorking (org:target "password", filename:.env), gists/GitLab, and trufflehog/gitleaks over public repos surface live API keys and tokens.
• Attack surface: enumerate subdomains with amass/subfinder plus certificate transparency (crt.sh), then resolve to find live hosts — never rely on a single source.
• People & email convention: LinkedIn + hunter.io to derive the username/email format and org chart for spraying and phishing.
• Document metadata: exiftool/FOCA on public PDFs/Office files leaks usernames, software versions, and internal paths.
• Confirm before reporting: validate that a leaked credential/asset is actually live (in-scope auth test, DNS resolves). Don't conclude "nothing found" until passive (CT logs, breach/paste sites) AND active (subdomain brute, GitHub dorks) sources are exhausted.

Prerequisites

• Familiarity with red teaming concepts and tools
• Access to a test or lab environment for safe execution
• Python 3.8+ with required dependencies installed
• Appropriate authorization for any testing activities

Objectives

• Enumerate the target organization's external attack surface (domains, IPs, cloud assets)
• Identify employees and their roles for social engineering targeting
• Discover leaked credentials, API keys, and sensitive documents
• Map the organization's technology stack and vendors
• Identify physical locations, office layouts, and access control details
• Build target profiles for spearphishing campaign development

Core Concepts

OSINT Categories

MITRE ATT&CK Mapping

• T1595.001 - Active Scanning: Scanning IP Blocks
• T1595.002 - Active Scanning: Vulnerability Scanning
• T1592 - Gather Victim Host Information
• T1589 - Gather Victim Identity Information
• T1590 - Gather Victim Network Information
• T1591 - Gather Victim Org Information
• T1593 - Search Open Websites/Domains
• T1594 - Search Victim-Owned Websites
• T1596 - Search Open Technical Databases

Workflow

Phase 1: Domain and Network Reconnaissance

1. Perform WHOIS lookups for target domains
2. Enumerate subdomains using Certificate Transparency logs, DNS brute-force, and web scraping
3. Identify IP ranges and ASN ownership
4. Scan for exposed services using Shodan/Censys
5. Check for cloud storage buckets (S3, Azure Blob, GCS)
6. Map CDN and hosting providers

Phase 2: Personnel and Social Intelligence

1. Enumerate employees via LinkedIn, company website, and conference speaker lists
2. Identify email naming conventions
3. Discover personal social media accounts of key targets
4. Map organizational hierarchy and reporting structure
5. Identify recently hired IT/security personnel
6. Check for conference presentations and technical publications

Phase 3: Credential and Data Leak Discovery

1. Search breach databases (Have I Been Pwned, DeHashed)
2. Check paste sites (Pastebin, GitHub Gists)
3. Search GitHub/GitLab for leaked secrets and API keys
4. Look for exposed configuration files and backups
5. Check for leaked internal documents via Google dorking

Phase 4: Technology Stack Identification

1. Analyze job postings for technology mentions
2. Use Wappalyzer/BuiltWith for web technology fingerprinting
3. Check for exposed admin panels and development environments
4. Identify VPN and remote access technologies
5. Map cloud services and SaaS applications

Tools and Resources

Validation Criteria

• Complete list of target domains and subdomains
• Employee list with roles and email addresses
• Technology stack identified
• Credential leak assessment completed
• Attack surface map documented
• OSINT report compiled for engagement team