Linux Log Analyzer

Linux Log Analyzer

Overview

Analyzes any Linux log content and produces a structured report: log type detection, event timeline, error/warning summary, security threat detection, anomaly flagging, and prioritized recommendations. Writes the full report to disk and prints only the key findings summary.

Language: Respond in the user's language. Report content in English.

When to Use

• User pastes log content and wants it analyzed
• User asks "what's wrong with my server?" and provides logs
• User wants to find brute force attacks, failed logins, unusual IPs
• User wants to understand why a service crashed or behaved unexpectedly
• User wants a summary of a long log file
• User asks about errors in nginx, sshd, Docker, cron, kernel logs

When NOT to Use

• User wants to set up centralized log management → use /linux-monitoring-setup
• User wants to write a script that parses logs → use /linux-shell-scriptor
• User wants to audit a config file → use /linux-config-auditor

Step 1 — Accept Input

Three input methods:

If the user describes a problem without pasting logs, ask:

"Paste the relevant log lines (or the full log file content) and I'll analyze them."

Step 2 — Detect Log Type

Auto-detect from content patterns. Never ask the user what log type it is unless completely ambiguous.

Step 3 — Analysis Framework

Run all analysis passes. Adjust depth to the log volume.

Pass 1 — Error & Warning Summary

Count and categorize by severity:

Pass 2 — Security Threat Detection

Always run for auth.log, syslog, nginx, ufw logs:

Pass 3 — Performance & Stability Issues

Pass 4 — Timeline of Notable Events

Extract the 10–20 most significant events in chronological order with timestamps.

Pass 5 — IP Intelligence

For any log containing IP addresses:

• List top 10 IPs by request/event count
• Flag IPs with >10 failed auth attempts
• Flag IPs that appear in both failures and successes (potential breach)
• Note any internal vs external IP patterns
• Classify IPs: private RFC1918 (10.x, 172.16–31.x, 192.168.x) vs public
• Flag known bad ranges (Tor exit nodes, common scanner ASNs like Shodan, Censys)

Pass 6 — Extended Analysis

Run these deeper passes when the log has enough data (>50 lines or contains meaningful time range):

Frequency & Spike Detection

• Bucket events by hour or minute and calculate the baseline rate
• Flag any window where the event rate is >3× the average — these are anomalous spikes
• Example output: "Error rate spiked 8× at 03:14 — 47 errors in 60 seconds vs. average of 6/min"

Cross-Event Correlation

Link related events that occurred close in time (within 60 seconds):

• SSH failure burst → followed by successful login = credential stuffing / breach attempt
• Service restart → followed by error spike = crash loop with side effects
• OOM kill → followed by slow query or timeout = resource starvation cascade
• UFW block spike → followed by auth failures = coordinated attack from same actor

Trend Analysis

If the log spans more than 1 hour:

• Is the error rate increasing, decreasing, or stable?
• Are any IPs showing escalating activity (first scans, then auth attempts, then exploits)?
• Any recurring patterns (same error every hour = likely a broken cron job)?

Filesystem & Kernel Extended Analysis (ext2/ext3/ext4, hardware)

Specifically for kern.log and syslog:

For each filesystem/hardware finding, include the exact recommended command to diagnose further.

HTTP / Web Log Pattern Analysis

For nginx and Apache access logs only:

• Status code distribution: count 2xx / 3xx / 4xx / 5xx — high 5xx = app problem, high 4xx = scanners/bad clients
• Top requested paths: list top 20 URLs — flag any that look like vulnerability scanning (/.env, /wp-login.php, /admin, /shell, /phpinfo.php)
• Response size anomalies: unusually large responses could indicate data exfiltration
• User-agent analysis: flag empty user-agents, known scanner agents (Nikto, sqlmap, masscan, zgrab), and headless browser signatures
• Request rate per IP per minute: flag IPs exceeding 60 req/min as potential DDoS or scrapers
• Slow requests: flag any request_time > 5s if log format includes it

Database Log Extended Analysis

For MySQL, PostgreSQL, MongoDB logs:

• List slow queries (>1s) with count and average duration
• Flag lock wait timeout — identifies transaction conflicts
• Flag too many connections — indicates connection pool exhaustion
• Flag deadlock found — requires query/index review
• Replication errors: Slave_IO_Running: No, replication lag

Step 4 — Write Report

Write the full report to ./log-analysis/report-<log-type>-<YYYYMMDD>.md in the current working directory.

Report structure:

# Log Analysis Report
**Log type:** [detected type]
**Lines analyzed:** [count]
**Time range:** [first timestamp] → [last timestamp]
**Generated:** [date]

## Executive Summary
[2–3 sentences: what was found, severity, most important action]

## Findings by Severity
### 🔴 Critical / Fatal ([count])
[table: timestamp | message | count]

### 🟠 Errors ([count])
[table: timestamp | message | count]

### 🟡 Warnings ([count])
[table: top entries]

## Security Events
[table: threat type | source IP | count | first seen | last seen | verdict]

## Top IPs
[table: IP | requests | failed auths | successful auths | verdict]

## Service / Stability Events
[list of crashes, restarts, OOM events with timestamps]

## Filesystem & Hardware Events
[ext4 errors, disk I/O errors, kernel panics, MCE errors — with recommended diagnostic commands]

## Extended Analysis
### Anomalous Spikes
[hourly event rate table — flag windows >3× average]

### Correlated Event Chains
[linked event sequences with timestamps and interpretation]

### Trend
[is the situation stable, improving, or worsening?]

### HTTP Pattern Analysis (if web log)
[status code distribution, top paths, scanner activity, suspicious user-agents]

### Database Analysis (if DB log)
[slow queries, lock waits, connection issues]

## Notable Events Timeline
[chronological list of the 10–20 most significant events]

## Recommendations
[numbered, prioritized list — most urgent first]

Step 5 — Chat Output

Print ONLY this after writing the report:

✅ Log analysis complete → ./log-analysis/report-[type]-[date].md

📊 Summary ([N] lines · [time range]):
   🔴 Critical:  [count]
   🟠 Errors:    [count]
   🟡 Warnings:  [count]
   🔵 Notable:   [count]

🔒 Security: [one line — e.g. "47 brute force attempts from 3.4.5.6 — IP should be blocked"]
⚡ Top issue: [one line — e.g. "nginx upstream timeout 23 times between 02:00–03:00"]

▶ Recommended actions:
  1. [most urgent action]
  2. [second action]
  3. [third action]

💡 Next: /linux-security-hardener to block attack vectors · /linux-config-auditor to fix misconfigs

If no issues were found:

✅ Log analysis complete → ./log-analysis/report-[type]-[date].md
   No critical issues found. [N] lines analyzed, [time range].

Quick Reference — Common Findings & Actions