Instructions
- Install dependencies:
pip install geoip2 user-agents
- Collect web server access logs in Combined Log Format (Apache) or Nginx default format.
- Parse each log entry extracting: IP, timestamp, method, URI, status code, response size, user-agent, referer.
- Apply detection rules:
- SQL injection:
UNION SELECT, OR 1=1, ' OR ', hex encoding patterns
- LFI/Path traversal:
../, /etc/passwd, /proc/self, php://filter
- XSS:
<script>, javascript:, onerror=, onload=
- Scanner signatures: nikto, sqlmap, dirbuster, gobuster, wfuzz user-agents
- Brute force: >50 POST requests to login endpoints from same IP in 5 minutes
- Enrich with GeoIP data and generate a prioritized findings report.
python scripts/agent.py --log-file /var/log/nginx/access.log --geoip-db GeoLite2-City.mmdb --output web_intrusion_report.json
Examples
Detect SQLi in URI
192.168.1.100 - - [15/Jan/2024:10:30:45 +0000] "GET /products?id=1' UNION SELECT username,password FROM users-- HTTP/1.1" 200 4532
Scanner User-Agent Detection
Nikto/2.1.6, sqlmap/1.7, DirBuster-1.0-RC1, gobuster/3.1.0
1---2name: analyzing-web-server-logs-for-intrusion3description: >-4 Parse Apache and Nginx access logs to detect SQL injection attempts, local file inclusion,5 directory traversal, web scanner fingerprints, and brute-force patterns. Uses regex-based6 pattern matching against OWASP attack signatures, GeoIP enrichment for source attribution,7 and statistical anomaly detection for request frequency and response size outliers.8domain: cybersecurity9subdomain: security-operations10tags: [analyzing, web, server, logs]11version: "1.0"12author: mahipal13license: Apache-2.014---1516## Instructions17181. Install dependencies: `pip install geoip2 user-agents`192. Collect web server access logs in Combined Log Format (Apache) or Nginx default format.203. Parse each log entry extracting: IP, timestamp, method, URI, status code, response size, user-agent, referer.214. Apply detection rules:22 - SQL injection: `UNION SELECT`, `OR 1=1`, `' OR '`, hex encoding patterns23 - LFI/Path traversal: `../`, `/etc/passwd`, `/proc/self`, `php://filter`24 - XSS: `<script>`, `javascript:`, `onerror=`, `onload=`25 - Scanner signatures: nikto, sqlmap, dirbuster, gobuster, wfuzz user-agents26 - Brute force: >50 POST requests to login endpoints from same IP in 5 minutes275. Enrich with GeoIP data and generate a prioritized findings report.2829```bash30python scripts/agent.py --log-file /var/log/nginx/access.log --geoip-db GeoLite2-City.mmdb --output web_intrusion_report.json31```3233## Examples3435### Detect SQLi in URI36```37192.168.1.100 - - [15/Jan/2024:10:30:45 +0000] "GET /products?id=1' UNION SELECT username,password FROM users-- HTTP/1.1" 200 453238```3940### Scanner User-Agent Detection41```42Nikto/2.1.6, sqlmap/1.7, DirBuster-1.0-RC1, gobuster/3.1.043```
autohandai