Osint

Open Source Intelligence (OSINT) Knowledge Base

OSINT collects publicly available information about targets without any direct interaction. This intelligence informs social engineering, credential attacks, and helps map the human attack surface alongside technical infrastructure.

1. Email Harvesting

theHarvester

# Comprehensive email harvesting
theHarvester -d <target> -b all -l 500 -f theharvester_<target>.html

# Specific sources
theHarvester -d <target> -b google,bing,linkedin,twitter -l 200

# Output formats
theHarvester -d <target> -b all -f theharvester_<target> --screenshot screenshots/

Manual Email Pattern Discovery

# Common email formats to test
# firstname.lastname@target.com
# firstnamelastname@target.com
# f.lastname@target.com
# firstname@target.com

# Verify email format via MX + SMTP (if in scope)
dig <target> MX +short

Email Analysis Points

• Naming convention: Determines brute-force pattern for credential stuffing
• Role-based emails: security@, admin@, devops@ reveal team structure
• Personal domains: Cross-reference with social media for password patterns
• Catch-all detection: Some domains accept all addresses → harder to enumerate

2. Employee & Organization Enumeration

LinkedIn Intelligence

• Search site:linkedin.com/in "<target company>" in Google
• Note: Direct LinkedIn scraping may violate ToS — use public search results
• Key data: Job titles, tech stack mentions, team sizes, recent hires

Organizational Mapping

Target Corp
├── Engineering (mentions: Kubernetes, Go, React)
│   ├── Platform Team (AWS, Terraform)
│   ├── Backend Team (Python, FastAPI)
│   └── Frontend Team (React, TypeScript)
├── Security
│   └── SOC Team (Splunk, CrowdStrike mentioned)
├── DevOps/SRE
│   └── (Jenkins, ArgoCD, Datadog mentioned)
└── IT
    └── (Okta, Jamf mentioned in job posts)

Tech Stack from Job Postings

Search job boards for:
"<target>" AND ("kubernetes" OR "terraform" OR "aws" OR "azure")
"<target>" AND ("react" OR "angular" OR "vue" OR "nextjs")
"<target>" AND ("python" OR "golang" OR "java" OR "rust")

3. GitHub & Code Repository OSINT

Organization Discovery

# GitHub org search
curl -s "https://api.github.com/orgs/<target>/repos?per_page=100" | \
    python3 -c "import sys,json; [print(r['full_name'], r.get('description','')) for r in json.load(sys.stdin)]"

# Search for secrets in public repos
# Google dork approach
# site:github.com "<target>" password | secret | api_key | token
# site:github.com "<target>.com" filename:.env

Secret Detection Patterns

# Using trufflehog (if available)
trufflehog github --org <target> --only-verified

# Using gitleaks (if available)
gitleaks detect --source /path/to/repo --report-path gitleaks_<target>.json

# Manual grep patterns in discovered repos
grep -rn "api[_-]key\|secret\|password\|token\|aws_access" --include="*.py" --include="*.js" --include="*.yaml" --include="*.env"

Key Findings from Code Repos

• Hardcoded credentials: API keys, database passwords, AWS keys
• Internal URLs: staging/dev environments, internal APIs
• Infrastructure as Code: Terraform/CloudFormation files reveal architecture
• CI/CD configs: .github/workflows/, Jenkinsfile reveal deployment pipeline
• Dependency files: package.json, requirements.txt reveal tech stack

4. Domain & IP Intelligence

Threat Intelligence

# VirusTotal domain report
curl -s "https://www.virustotal.com/api/v3/domains/<target>" \
    -H "x-apikey: <VT_API_KEY>" | python3 -m json.tool

# Shodan host info (if API key available)
curl -s "https://api.shodan.io/shodan/host/<IP>?key=<SHODAN_KEY>" | python3 -m json.tool

# AbuseIPDB check
curl -s "https://api.abuseipdb.com/api/v2/check?ipAddress=<IP>" \
    -H "Key: <ABUSE_KEY>" -H "Accept: application/json"

Domain Reputation

• VirusTotal: Malware associations, passive DNS, URL scan history
• Shodan/Censys: Exposed services, historical banners, SSL certs
• SecurityTrails: Historical DNS records, IP history
• URLScan.io: Live page rendering, resource loading, redirects

Wayback Machine

# Historical snapshots
curl -s "https://web.archive.org/cdx/search/cdx?url=*.example.com/*&output=text&fl=original&collapse=urlkey" | \
    sort -u > wayback_<target>.txt

# Look for old admin panels, login pages, API docs
grep -iE "(admin|login|api|swagger|debug|config)" wayback_<target>.txt

5. Breach & Credential Data

Approach (Ethical Boundaries)

• Check Have I Been Pwned API for breach exposure (with authorization)
• Document which breaches may contain target employee credentials
• Never access or download actual breach databases without explicit authorization
• Focus on: Which breaches? What data types exposed? Credential reuse risk?

HIBP API Check

# Check domain breach status (requires API key)
curl -s "https://haveibeenpwned.com/api/v3/breaches" \
    -H "hibp-api-key: <KEY>" | python3 -c "
import sys, json
for b in json.load(sys.stdin):
    if '<target>' in b.get('Domain','').lower():
        print(f\"{b['Name']}: {b['BreachDate']} - {b['PwnCount']:,} records\")
        print(f\"  Data: {', '.join(b['DataClasses'])}\")"

Credential Intelligence (What to Document)

• Number of breaches involving target domain
• Types of data exposed (passwords, hashes, emails, PII)
• Breach dates (recent = higher risk of valid credentials)
• Password reuse likelihood across services

6. Social Media Intelligence

Platform-Specific Searches

Twitter/X: from:<target_handle> | "@<target>" filter:links
Reddit: site:reddit.com "<target company>"
Glassdoor: Company reviews reveal internal culture, tech stack, complaints
Crunchbase: Funding, acquisitions, key personnel

Metadata from Social Media

• Photos: EXIF data (GPS coordinates, device info, timestamps)
• Documents: PDF metadata (author names, software versions, internal paths)
• Posts: Technology mentions, frustration with tools (vulnerability hints)

7. Google Dorking (Advanced)

Target-Specific Dorks

# Exposed documents
site:<target> filetype:pdf | filetype:doc | filetype:xls
site:<target> filetype:sql | filetype:bak | filetype:log

# Admin/config exposure
site:<target> inurl:admin | inurl:login | inurl:dashboard
site:<target> intitle:"index of" | intitle:"directory listing"

# Error messages / debug info
site:<target> "error" | "exception" | "stack trace" | "debug"
site:<target> "not for distribution" | "confidential" | "internal use only"

# Third-party integrations
"<target>.com" site:trello.com | site:notion.so | site:pastebin.com
"<target>.com" site:stackoverflow.com "api" | "key" | "secret"

8. Workflow: OSINT Sequence

1. Email Harvesting → theHarvester, manual pattern discovery
2. Employee Enumeration → LinkedIn, job postings, org chart mapping
3. Code Repository Search → GitHub org, secret scanning, IaC review
4. Domain Intelligence → VirusTotal, Shodan, SecurityTrails
5. Breach Assessment → HIBP, breach database scope
6. Social Media → Platform searches, metadata extraction
7. Google Dorking → Advanced searches for exposed data
8. Wayback Machine → Historical URL analysis
9. Synthesis → Combine with technical recon, build target profile

9. Output Files

./
├── theharvester_<target>.html     # Email harvesting results
├── emails_<target>.txt            # Cleaned email list
├── org_chart_<target>.md          # Organizational mapping
├── github_secrets_<target>.txt    # Code repo findings
├── wayback_<target>.txt           # Historical URLs
├── breach_assessment_<target>.md  # Breach exposure summary
└── osint_<target>_summary.md      # Consolidated OSINT report

10. Ethical Boundaries

• Always passive: OSINT must not involve direct target interaction
• Public data only: Only use information available in public sources
• No impersonation: Do not create fake accounts or social engineer employees
• Breach data: Document exposure risk, never access raw breach dumps without authorization
• Privacy: Report findings that are security-relevant, not personal embarrassments