Detecting Port Scanning With Fail2ban

Detecting Port Scanning with Fail2ban

When to Use

• Automatically blocking IP addresses that perform port scans against internet-facing servers
• Defending SSH, HTTP, FTP, and other services against brute force attacks with automated IP banning
• Creating custom detection filters for organization-specific attack patterns in log files
• Reducing noise from automated scanning bots before traffic reaches IDS/IPS for deeper analysis
• Implementing defense-in-depth by adding host-based automated response to network monitoring

Do not use as the sole network security control, for protecting against distributed attacks from many source IPs, or as a replacement for proper firewall rules and network segmentation.

Detection Gaps & Validation

Fail2ban is per-source and log-driven, so it misses exactly the scans attackers use to stay quiet:

• Slow scans evade findtime: the recent --hitcount 20 --seconds 10 iptables rule and a findtime = 60 jail never trigger on nmap -T1 (≈1 probe/15s). Widen findtime or add a long-window jail; "no bans" does not mean "no scan."
• Distributed scans: Fail2ban bans one IP at a time and cannot correlate a scan spread across a /24 of source hosts. Pair it with a network IDS for distributed recon.
• SYN scans may never log: -sS to a closed port only matches if your PORTSCAN chain actually logs it; an ACCEPT earlier in INPUT means the probe is never written and Fail2ban sees nothing.
• Backend mismatch: backend = systemd reads the journal, not /var/log/kern.log; if iptables logs go only to the file, the jail silently parses zero lines.

How to confirm detection works: run fail2ban-regex /var/log/kern.log /etc/fail2ban/filter.d/portscan.conf and verify a non-zero match count, then nmap -sS -p1-1000 from an authorized host and confirm the IP appears in fail2ban-client status portscan. A jail that loads with 0 matched lines is a coverage gap, not a clean network.

Prerequisites

• Fail2ban 0.11+ installed (fail2ban-client --version)
• Root/sudo access for iptables/nftables manipulation
• Services logging connection attempts to parseable log files (syslog, auth.log, access.log)
• iptables or nftables installed and operational as the host firewall
• Optional: SMTP server for email notifications on ban events

Workflow

Step 1: Install and Configure Fail2ban

# Install Fail2ban
sudo apt install -y fail2ban

# Create local configuration (never edit jail.conf directly)
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

# Configure global defaults
sudo tee /etc/fail2ban/jail.local << 'EOF'
[DEFAULT]
# Ban duration (1 hour default, escalates for repeat offenders)
bantime = 3600
# Detection window
findtime = 600
# Max failures before ban
maxretry = 5
# Ban action using iptables
banaction = iptables-multiport
banaction_allports = iptables-allports
# Email notifications
destemail = security@example.com
sender = fail2ban@example.com
mta = sendmail
action = %(action_mwl)s

# Ignore internal networks
ignoreip = 127.0.0.1/8 ::1 10.10.0.0/16

# Use systemd journal backend where available
backend = systemd

[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 7200
findtime = 300

[sshd-ddos]
enabled = true
port = ssh
filter = sshd-ddos
logpath = /var/log/auth.log
maxretry = 6
bantime = 3600
EOF

Step 2: Create Custom Port Scan Detection Filter

# Create iptables logging rule for dropped connections
sudo iptables -N PORTSCAN
sudo iptables -A PORTSCAN -j LOG --log-prefix "PORTSCAN_DETECTED: " --log-level 4
sudo iptables -A PORTSCAN -j DROP

# Log SYN packets to closed ports (indicates scanning)
sudo iptables -A INPUT -p tcp --tcp-flags SYN,ACK,FIN,RST SYN -m state --state NEW \
  -m recent --name portscan --set
sudo iptables -A INPUT -p tcp --tcp-flags SYN,ACK,FIN,RST SYN -m state --state NEW \
  -m recent --name portscan --rcheck --seconds 10 --hitcount 20 -j PORTSCAN

# Create Fail2ban filter for port scanning
sudo tee /etc/fail2ban/filter.d/portscan.conf << 'EOF'
[Definition]
# Match iptables port scan log entries
failregex = PORTSCAN_DETECTED: .* SRC=<HOST> DST=\S+ .* DPT=\d+
ignoreregex =
datepattern = {^LN-BEG}
EOF

# Create Fail2ban filter for Nmap detection via kernel logs
sudo tee /etc/fail2ban/filter.d/nmap-scan.conf << 'EOF'
[Definition]
# Detect rapid connection attempts to multiple ports from same source
failregex = kernel: \[.*\] PORTSCAN_DETECTED: .* SRC=<HOST>
            iptables: .* PORTSCAN .* SRC=<HOST>
ignoreregex =
datepattern = {^LN-BEG}
EOF

# Create filter for HTTP scanning/probing
sudo tee /etc/fail2ban/filter.d/http-scan.conf << 'EOF'
[Definition]
# Detect scanners probing for common vulnerabilities
failregex = ^<HOST> .* "(GET|POST|HEAD) /(wp-login|wp-admin|phpmyadmin|admin|.env|xmlrpc|wp-content/uploads).*" (403|404|444)
            ^<HOST> .* "(GET|POST) /.*\.(php|asp|aspx|jsp|cgi)\?.*" (403|404)
            ^<HOST> .* "() .*" 400
            ^<HOST> .* "(GET|POST) /.*" 400
ignoreregex =
datepattern = {^LN-BEG}
EOF

Step 3: Configure Jail for Port Scanning

# Add port scan jails to jail.local
sudo tee -a /etc/fail2ban/jail.local << 'EOF'

[portscan]
enabled = true
filter = portscan
logpath = /var/log/kern.log
maxretry = 10
findtime = 60
bantime = 86400
banaction = iptables-allports
action = %(action_mwl)s

[nmap-scan]
enabled = true
filter = nmap-scan
logpath = /var/log/kern.log
maxretry = 5
findtime = 30
bantime = 86400
banaction = iptables-allports
action = %(action_mwl)s

[http-scan]
enabled = true
filter = http-scan
logpath = /var/log/nginx/access.log
maxretry = 10
findtime = 300
bantime = 3600
banaction = iptables-multiport
port = http,https

[recidive]
enabled = true
filter = recidive
logpath = /var/log/fail2ban.log
bantime = 604800
findtime = 86400
maxretry = 3
banaction = iptables-allports
action = %(action_mwl)s
EOF

Step 4: Configure Advanced Ban Actions

# Create custom action that blocks and sends webhook notification
sudo tee /etc/fail2ban/action.d/iptables-webhook.conf << 'EOF'
[Definition]
actionstart = <iptables> -N f2b-<name>
              <iptables> -A f2b-<name> -j RETURN
              <iptables> -I <chain> -p <protocol> -j f2b-<name>

actionstop = <iptables> -D <chain> -p <protocol> -j f2b-<name>
             <iptables> -F f2b-<name>
             <iptables> -X f2b-<name>

actioncheck = <iptables> -n -L <chain> | grep -q 'f2b-<name>[ \t]'

actionban = <iptables> -I f2b-<name> 1 -s <ip> -j <blocktype>
            curl -s -X POST "<webhook_url>" \
              -H "Content-Type: application/json" \
              -d '{"text":"[Fail2ban] Banned <ip> from <name> jail (failures: <failures>)"}'

actionunban = <iptables> -D f2b-<name> -s <ip> -j <blocktype>

[Init]
chain = INPUT
blocktype = DROP
webhook_url = https://hooks.slack.com/services/XXXX/YYYY/ZZZZ
EOF

# Create escalating ban action for repeat offenders
sudo tee /etc/fail2ban/action.d/escalating-ban.conf << 'EOF'
[Definition]
actionban = <iptables> -I f2b-<name> 1 -s <ip> -j DROP
            echo "$(date) BAN <ip> jail=<name> failures=<failures> bantime=<bantime>" >> /var/log/fail2ban-bans.log

actionunban = <iptables> -D f2b-<name> -s <ip> -j DROP
              echo "$(date) UNBAN <ip> jail=<name>" >> /var/log/fail2ban-bans.log
EOF

Step 5: Test and Validate Detection

# Restart Fail2ban
sudo systemctl restart fail2ban

# Verify jails are active
sudo fail2ban-client status
sudo fail2ban-client status sshd
sudo fail2ban-client status portscan

# Test the port scan filter with a regex check
sudo fail2ban-regex /var/log/kern.log /etc/fail2ban/filter.d/portscan.conf

# Test the HTTP scan filter
sudo fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/http-scan.conf

# Simulate a port scan from a test machine (authorized)
# From the test machine:
nmap -sS -p 1-1000 <target_ip>

# Verify the scanner gets banned
sudo fail2ban-client status portscan
# Should show the test IP in the banned list

# Check iptables for the ban rule
sudo iptables -L f2b-portscan -n

# Unban the test IP
sudo fail2ban-client set portscan unbanip <test_ip>

Step 6: Monitor and Maintain

# View real-time ban activity
sudo tail -f /var/log/fail2ban.log | grep -E "Ban|Unban"

# Generate daily summary report
sudo tee /usr/local/bin/fail2ban-report.sh << 'SCRIPT'
#!/bin/bash
echo "=== Fail2ban Daily Report $(date) ==="
echo ""
echo "Active Jails:"
sudo fail2ban-client status | grep "Jail list"
echo ""
echo "Currently Banned IPs:"
for jail in $(sudo fail2ban-client status | grep "Jail list" | sed 's/.*://;s/,//g'); do
    count=$(sudo fail2ban-client status "$jail" | grep "Currently banned" | awk '{print $NF}')
    if [ "$count" -gt 0 ]; then
        echo "  $jail: $count banned"
        sudo fail2ban-client status "$jail" | grep "Banned IP"
    fi
done
echo ""
echo "Last 24 hours - Ban count by jail:"
grep "Ban " /var/log/fail2ban.log | grep "$(date +%Y-%m-%d)" | awk '{print $NF}' | sort | uniq -c | sort -rn
SCRIPT
chmod +x /usr/local/bin/fail2ban-report.sh

# Schedule daily report
echo "0 8 * * * root /usr/local/bin/fail2ban-report.sh | mail -s 'Fail2ban Report' security@example.com" | sudo tee /etc/cron.d/fail2ban-report

# Persist iptables rules across reboots
sudo apt install iptables-persistent
sudo netfilter-persistent save

Key Concepts

Tools & Systems

• Fail2ban 0.11+: Log-parsing intrusion prevention framework that bans IP addresses based on pattern matching across any log file
• iptables/nftables: Linux kernel firewall used by Fail2ban ban actions to block offending IP addresses at the network layer
• fail2ban-regex: Testing utility for validating filter regular expressions against actual log files before deploying to production
• fail2ban-client: Command-line management tool for querying jail status, manually banning/unbanning IPs, and reloading configuration
• rsyslog/syslog-ng: System logging daemons that generate the log files Fail2ban monitors for attack detection

Common Scenarios

Scenario: Defending a Public-Facing Web Server Against Automated Scanning

Context: A company runs a public web server that receives thousands of automated scan attempts daily from bots probing for vulnerable paths (/wp-admin, /phpmyadmin, /.env). The security team wants to automatically block scanners while allowing legitimate traffic. The server runs Nginx on Ubuntu 22.04.

Approach:

1. Install Fail2ban and configure it to monitor Nginx access logs for scanning patterns (404/403 responses to known vulnerability paths)
2. Create a custom http-scan filter matching common scanner signatures and vulnerability probing URIs
3. Set maxretry to 10 within a 5-minute findtime, with a 1-hour bantime for first offense
4. Enable the recidive jail to escalate ban duration to 7 days for repeat offenders
5. Configure webhook notifications to Slack for real-time visibility of banning activity
6. Add iptables logging rules for SYN packets to closed ports to detect port scanning
7. Create a daily report script showing banned IPs, attack patterns, and geographic distribution

Pitfalls:

• Setting maxretry too low (e.g., 1-2), causing legitimate users who mistype URLs to get banned
• Not whitelisting monitoring systems (Nagios, UptimeRobot) that may trigger filters with their health checks
• Forgetting to persist iptables rules, losing all bans after a reboot
• Not testing filters with fail2ban-regex before deploying, resulting in no matches or excessive false positives

Output Format

## Fail2ban Port Scan Defense Report

**Server**: web-prod-01 (203.0.113.50)
**Reporting Period**: 2024-03-15 00:00 to 2024-03-16 00:00 UTC

### Active Jails

| Jail | Filter | Max Retry | Ban Time | Currently Banned |
|------|--------|-----------|----------|------------------|
| sshd | sshd | 3 | 2 hours | 12 IPs |
| portscan | portscan | 10 | 24 hours | 47 IPs |
| http-scan | http-scan | 10 | 1 hour | 89 IPs |
| recidive | recidive | 3 | 7 days | 8 IPs |

### 24-Hour Summary
- Total ban events: 347
- Unique IPs banned: 156
- Top attacking country: CN (67 IPs), RU (34 IPs), US (21 IPs)
- Most targeted service: HTTP scanning (214 bans)
- Recidive escalations: 8 IPs banned for 7 days

### Top 5 Banned IPs
| IP Address | Jail | Ban Count | First Seen | Last Seen |
|------------|------|-----------|------------|-----------|
| 45.33.32.156 | portscan | 12 | 00:15 | 23:47 |
| 198.51.100.23 | http-scan | 8 | 02:30 | 18:22 |
| 203.0.113.100 | sshd | 6 | 05:12 | 21:33 |