Web Pentesting

Web Application Penetration Testing

A comprehensive methodology for security testing web applications. This skill guides you through systematic enumeration, vulnerability discovery, and assessment of web services.

When to Use This Skill

Use this skill when:

• Assessing web applications for security vulnerabilities
• Performing bug bounty hunting on web targets
• Testing HTTP/HTTPS services (ports 80/443)
• Enumerating web technologies and versions
• Discovering hidden endpoints, files, or parameters
• Conducting CTF web challenges
• Performing security audits on web infrastructure

Methodology Overview

Follow this systematic approach for each target domain, subdomain, or IP:

1. Technology Identification - Identify web server, frameworks, and versions
2. Initial Checks - Check robots.txt, sitemap, error pages, SSL/TLS
3. Spidering - Crawl to discover all paths, files, and parameters
4. Directory Brute-Forcing - Find hidden directories and files
5. Backup File Discovery - Look for exposed backups and configs
6. Parameter Discovery - Find hidden form/query parameters
7. Vulnerability Testing - Test discovered endpoints for vulnerabilities

1. Technology Identification

Identify Web Server and Technologies

Check HTTP headers and use automated tools to identify technologies:

# Basic HTTP request to see headers
curl -I https://target.com

# WhatWeb - technology identification
whatweb -a 1 https://target.com  # Stealthy
whatweb -a 3 https://target.com  # Aggressive

# Web technology detection
webtech -u https://target.com

# BuiltWith API (if available)
# https://builtwith.com/

Check for Known Vulnerabilities

Once you identify the technology and version:

1. Search for CVEs related to the specific version
2. Check if the technology has known default credentials
3. Look for specialized scanners for that technology

Check for WAF (Web Application Firewall)

# WAF detection tools
wafw00f https://target.com
whatwaf https://target.com

# Nmap WAF detection
nmap --script http-waf-detect https://target.com

Technology-Specific Tricks

If you identify specific technologies, apply targeted techniques:

• WordPress: Use wpscan, check for plugin vulnerabilities
• Drupal: Use droopescan, check for known exploits
• Joomla: Use joomscan, check for component vulnerabilities
• Apache/Nginx: Check for misconfigurations, directory traversal
• PHP: Look for file inclusion, code execution vulnerabilities
• Git: If .git is exposed, extract repository contents
• S3 Buckets: Check for public read/write permissions

2. Initial Checks

Default Information Pages

Check these common paths for information disclosure:

/robots.txt
/sitemap.xml
/crossdomain.xml
/clientaccesspolicy.xml
/.well-known/

Force Error Pages

Web servers may reveal information through error responses:

# Access fake pages to trigger errors
curl https://target.com/whatever_fake.php
curl https://target.com/nonexistent.aspx

# Add special characters to trigger errors
curl -H "Cookie: test=[]" https://target.com
curl "https://target.com/page?param=[[]]"

# Try different HTTP verbs
curl -X PATCH https://target.com
curl -X DEBUG https://target.com
curl -X FAKE https://target.com

SSL/TLS Vulnerability Check

# Comprehensive SSL/TLS testing
testssl.sh --htmlfile https://target.com:443

# Alternative tools
sslscan target.com:443
sslyze --regular target.com:443

Critical checks:

• Is HTTPS enforced? (MitM vulnerability if not)
• Are passwords sent over HTTP? (High severity)
• Are there weak cipher suites?

3. Spidering

Spider the application to discover all accessible paths, files, and parameters.

Spidering Tools

# Katana - modern, fast spider
katana -u https://target.com -J -d 3 -s /tmp/spider-output.txt

# Feroxbuster - content discovery
feroxbuster -u https://target.com -w /usr/share/wordlists/dirb/common.txt

# Gospider - HTML spider with external sources
gospider -S https://target.com -d 3 -t 10 -o /tmp/gospider-output.txt

# LinkFinder - extract paths from JavaScript files
python3 LinkFinder.py -d 3 -i /tmp/page.html -o /tmp/linkfinder-output

# gau - get all urls from external sources
gau https://target.com > /tmp/gau-urls.txt

# ParamSpider - find URLs with parameters
paramspider -d target.com -o /tmp/params.txt

JavaScript Analysis

JavaScript files often contain hidden endpoints and API keys:

# Extract URLs from JS files
python3 LinkFinder.py -f /path/to/file.js -o /tmp/js-endpoints

# JSluice - extract secrets and URLs from JS
jsluice /path/to/file.js

# SecretFinder - find API keys in JS
python3 SecretFinder.py -f /path/to/file.js

# Deobfuscate JavaScript
# Use: https://lelinhtinh.github.io/de4js/

4. Directory Brute-Forcing

Systematically discover hidden directories and files.

Brute-Force Tools

# Gobuster - fast, supports auto-signed certs
gobuster dir -w /usr/share/wordlists/dirb/common.txt -u https://target.com
gobuster dir -w /usr/share/wordlists/dirb/big.txt -u https://target.com -r  # recursive

# Feroxbuster - very fast, recursive
feroxbuster -u https://target.com -w /usr/share/wordlists/dirb/big.txt -r

# Dirsearch - Python-based, recursive
dirsearch -u https://target.com -w /usr/share/wordlists/dirb/common.txt -r

# ffuf - fast fuzzer
ffuf -w /usr/share/wordlists/dirb/big.txt -u https://target.com/FUZZ

# wfuzz - flexible fuzzer
wfuzz -w /usr/share/wordlists/dirb/common.txt https://target.com/FUZZ

Recommended Wordlists

• /usr/share/wordlists/dirb/common.txt
• /usr/share/wordlists/dirb/big.txt
• /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt
• https://github.com/danielmiessler/SecLists/tree/master/Discovery/Web-Content
• https://wordlists.assetnote.io

Important: Recursive Discovery

Any new directory discovered should be brute-forced recursively. This is critical for finding deeply nested resources.

5. Backup File Discovery

Look for exposed backup files, configuration files, and sensitive data.

Common Backup Extensions

Check these variations for each discovered file:

file.ext~
#file.ext#
~file.ext
file.ext.bak
file.ext.tmp
file.ext.old
file.bak
file.tmp
file.old
file.ext.swp
file.ext.orig

Automated Backup Checking

# bfac - backup file checker
bfac -u https://target.com -w /path/to/wordlist.txt

# Manual check for common files
curl https://target.com/.env
curl https://target.com/.git/config
curl https://target.com/config.php.bak
curl https://target.com/wp-config.php.bak

Sensitive Files to Check

.env
.git/
.git/config
.git/HEAD
.htaccess
.htpasswd
config.php
config.php.bak
wp-config.php
wp-config.php.bak
web.config
application.properties

6. Parameter Discovery

Find hidden form and query parameters that may be vulnerable.

Parameter Discovery Tools

# Arjun - discover hidden parameters
arjun -i https://target.com -o /tmp/arjun-output.txt

# Parameth - parameter discovery
parameth -u https://target.com

# x8 - parameter discovery
x8 -u https://target.com

# Burp Param Miner (if using Burp Suite)

Parameter Wordlists

• https://github.com/s0md3v/Arjun/tree/master/arjun/db
• https://github.com/PortSwigger/param-miner/blob/master/resources/params
• https://wordlists.assetnote.io/parameters_top_1m

7. Special Findings

.git Repository Exposure

If you find a .git directory exposed:

# Clone the exposed git repository
git clone file://https://target.com/.git /tmp/leaked-git
cd /tmp/leaked-git
git log --all
git show HEAD

API Key Discovery

If you find API keys, check their permissions:

# Google Maps API key checker
gmapapiscanner --key AIzaSy...

# General API key tools
trufflehog --regex --entropy 4.5 /path/to/files/
keyhacks

S3 Bucket Discovery

While spidering, look for S3 bucket references and check permissions:

# Check if bucket is public
aws s3 ls s3://bucket-name/

# Check bucket policy
aws s3api get-bucket-policy --bucket bucket-name

502 Proxy Error (SSRF)

If you see 502 errors, test for SSRF:

# Send request with Host header to test proxy
curl -H "Host: google.com" https://target.com

# If proxy forwards, you've found SSRF

NTLM Authentication Info Disclosure

# Send NTLM header to get server info
curl -H "Authorization: NTLM TlRMTVNTUAABAAAAB4IIAAAAAAAAAAAAAAAAAAAAAAA=" https://target.com

# Nmap automation
nmap --script http-ntlm-info https://target.com

8. Vulnerability Testing

After enumeration, test discovered endpoints for vulnerabilities:

Common Vulnerability Types

1. SQL Injection - Test all parameters with SQL payloads
2. XSS (Cross-Site Scripting) - Test input fields and parameters
3. Command Injection - Test for OS command execution
4. File Inclusion - Test for LFI/RFI vulnerabilities
5. Authentication Bypass - Test login mechanisms
6. Authorization Issues - Test for IDOR, privilege escalation
7. SSRF - Test for server-side request forgery
8. XXE - Test XML parsing endpoints
9. CSRF - Test state-changing operations
10. Insecure Direct Object References - Test for IDOR

Automated Scanners

# Nikto - web vulnerability scanner
nikto -h https://target.com

# Wapiti - web vulnerability scanner
wapiti -u https://target.com

# Nuclei - template-based scanner
nuclei -u https://target.com

# ZAP - comprehensive scanner
# Use ZAP API or GUI

# Puff - client-side vulnerability fuzzer
node puff.js -w ./wordlist-examples/xss.txt -u "https://target.com/?query=FUZZ"

CMS-Specific Scanners

# WordPress
wpscan --url https://target.com --enumerate ap,at,cb,dbe
wpscan --url https://target.com --enumerate u,tt,t,vp --passwords /path/to/passwords.txt

# Drupal
droopescan scan -t https://target.com

# Joomla
joomscan -u https://target.com

# General CMS
cmsmap -f W -F -d https://target.com  # WordPress
cmsmap -f J -F -d https://target.com  # Joomla
cmsmap -f D -F -d https://target.com  # Drupal

9. Monitoring

Monitor pages for changes that might indicate new vulnerabilities:

# changedetection.io
# https://github.com/dgtlmoon/changedetection.io

# Monitor specific pages for modifications
# Useful for long-term bug bounty programs

Quick Reference Commands

Rapid Web Assessment

# Technology identification
whatweb -a 3 https://target.com

# Quick vulnerability scan
nikto -h https://target.com

# Directory enumeration
gobuster dir -w /usr/share/wordlists/dirb/common.txt -u https://target.com

# Spider the site
katana -u https://target.com -J -d 3

# Check for exposed .git
curl https://target.com/.git/config

# Check for .env file
curl https://target.com/.env

WordPress Specific

# Full WordPress assessment
wpscan --url https://target.com --enumerate ap,at,cb,dbe,u,tt,t,vp --passwords /path/to/passwords.txt

# Brute force login
hydra -l admin -P /path/to/passwords.txt target.com -V http-form-post '/wp-login.php:log=^USER^&pwd=^PASS^&wp-submit=Log In&testcookie=1:S=Location'

Drupal Specific

# Drupal enumeration
droopescan scan -t https://target.com

# Get low-hanging fruit
git clone https://github.com/immunIT/drupwn.git
cd drupwn && python3 drupwn.py -u https://target.com

Best Practices

1. Always enumerate first - Don't jump to exploitation without understanding the target
2. Document everything - Keep track of discovered endpoints, parameters, and findings
3. Test recursively - Any new directory should be brute-forced
4. Check backups - Backup files often contain sensitive information
5. Analyze JavaScript - JS files frequently contain hidden endpoints and API keys
6. Use multiple tools - Different tools find different things
7. Manual testing - Automated tools miss context-specific vulnerabilities
8. Respect scope - Only test targets within your authorization

Output Format

When reporting findings, use this structure:

# Web Security Assessment Report

## Target
- URL: https://target.com
- IP: x.x.x.x
- Date: YYYY-MM-DD

## Technology Stack
- Web Server: Apache 2.4.41
- CMS: WordPress 5.8.2
- Plugins: [list]
- Themes: [list]

## Discovered Endpoints
- /admin/
- /wp-admin/
- /api/v1/
- [list all discovered paths]

## Vulnerabilities Found
### Critical
- [vulnerability description]
- [proof of concept]
- [impact]

### High
- [vulnerability description]

### Medium
- [vulnerability description]

### Low
- [vulnerability description]

## Recommendations
- [remediation steps]

Scripts

Use the bundled scripts for common tasks:

• scripts/quick-web-scan.sh - Rapid technology identification and vulnerability scanning
• scripts/directory-bruteforce.sh - Systematic directory enumeration
• scripts/backup-file-check.sh - Check for exposed backup files
• scripts/parameter-discovery.sh - Find hidden parameters
• scripts/tech-identification.sh - Comprehensive technology detection

Run scripts with:

./scripts/quick-web-scan.sh https://target.com
./scripts/directory-bruteforce.sh https://target.com

Notes

• This methodology assumes you have authorization to test the target
• Always follow responsible disclosure practices
• Document all findings with evidence
• Consider the impact of your testing on production systems
• Some automated scans may be noisy - coordinate with target owners
• For bug bounty programs, follow the program's specific rules and scope