Log Analysis Tools

Log Analysis Tools Skill

Purpose: Fast, efficient log file analysis with framework-aware patterns and intelligent pruning

🎨 VISUAL OUTPUT FORMATTING

CRITICAL: All log-analysis-tools output MUST use the colored-output formatter skill!

bash .claude/skills/colored-output/color.sh skill-header "log-analysis-tools" "Analyzing log files..."
bash .claude/skills/colored-output/color.sh progress "" "Parsing Laravel logs"
bash .claude/skills/colored-output/color.sh info "" "Found 15 errors in last 24h"
bash .claude/skills/colored-output/color.sh success "" "Analysis complete"

🎯 Multi-Framework Support

Framework Detection & Log Locations

Framework-Specific Patterns

Laravel Error Patterns:

^\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\] \w+\.(ERROR|WARNING|CRITICAL)

CodeIgniter Patterns:

^(ERROR|WARNING|CRITICAL) - \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}

Next.js Patterns:

^\s*(error|warn|info)\s+-\s+

Express/Node Patterns:

^\[\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z\]\s+(error|warn)

🚀 Auto-Activation Triggers

Primary Keywords:

• "check logs", "view logs", "tail logs", "show logs"
• "find errors", "log errors", "error log", "access log"
• "laravel log", "codeigniter log", "next.js log"
• "debug production", "production errors"
• "prune logs", "clean old logs", "archive logs"

Context-Aware Triggers:

• User mentions debugging → Auto-suggest log analysis
• User mentions "500 error" → Auto-open error logs
• User mentions "slow performance" → Check slow query logs
• User says "production issue" → Tail production logs

🖥️ Platform Compatibility

Supported Platforms

Required Dependencies

Optional Dependencies (Enhanced Features)

Tool Availability by Platform

macOS (via Homebrew)

# Install all tools (recommended)
brew install lnav fd ripgrep bat

# Or install individually
brew install lnav      # Advanced log viewer
brew install fd        # Fast file finding
brew install ripgrep   # Fast text search
brew install bat       # Syntax-highlighted cat

Linux (Ubuntu/Debian)

# Install all tools (recommended)
sudo apt update
sudo apt install lnav fd-find ripgrep bat

# Or install individually
sudo apt install lnav      # Advanced log viewer
sudo apt install fd-find   # Fast file finding (note: fd-find on Ubuntu)
sudo apt install ripgrep   # Fast text search
sudo apt install bat       # Syntax-highlighted cat

Linux (RHEL/CentOS/Fedora)

# Enable EPEL repository first (RHEL/CentOS)
sudo yum install epel-release

# Install tools
sudo yum install lnav ripgrep bat
sudo dnf install fd-find  # Fedora

# Or use cargo for latest versions
cargo install lnav fd-find ripgrep bat

Windows WSL

# Use Ubuntu/Debian commands (recommended)
# Or use Windows Package Manager
winget install BurntSushi.ripgrep.MSVC
winget install sharkdp.fd
winget install sharkdp.bat

# lnav not available natively - use WSL

Graceful Fallback Chain

The script automatically falls back to standard tools when enhanced tools are unavailable:

lnav → bat → less → cat (viewing)
fd → find (file finding)
ripgrep → grep (searching)
bat → cat (syntax highlighting)

Example:

# With all tools installed (fastest)
lnav storage/logs/laravel.log

# Without lnav (falls back to bat)
bat --paging=always -l log storage/logs/laravel.log

# Without bat (falls back to less)
less storage/logs/laravel.log

# Minimal (always works)
cat storage/logs/laravel.log

Dependency Checking

The script checks dependencies on startup and provides helpful install instructions:

$ bash log-tools.sh view

⚠️  Missing optional tools (enhanced features):
  - lnav
  - fd
  - ripgrep

Install for better performance:
  brew install lnav fd ripgrep  # macOS
  sudo apt install lnav fd-find ripgrep  # Linux

📝 Falling back to standard tools (slower but functional)

Performance Impact

Recommendation: Install at least lnav and ripgrep for optimal experience (90% of performance gains).

📊 Commands

1. View Logs (Framework-Aware)

Auto-detect framework and open appropriate logs:

# Laravel
bash .claude/skills/log-analysis-tools/log-tools.sh view

# Auto-detects:
# → composer.json exists → Laravel → storage/logs/laravel.log
# → Runs: lnav storage/logs/laravel.log

Manual framework specification:

bash .claude/skills/log-analysis-tools/log-tools.sh view laravel
bash .claude/skills/log-analysis-tools/log-tools.sh view codeigniter
bash .claude/skills/log-analysis-tools/log-tools.sh view nextjs

Features:

• ✅ Syntax highlighting by log level
• ✅ Auto-scroll to latest entries
• ✅ Press 'e' for errors only
• ✅ Press 't' to go to specific timestamp
• ✅ Press '/' to search

2. Find Errors (Last 24 Hours)

Extract only errors from today:

bash .claude/skills/log-analysis-tools/log-tools.sh errors

# Shows:
# - ERROR level entries from last 24 hours
# - Stack traces included
# - Grouped by error type
# - Count of each error

Output Example:

🔍 Errors in storage/logs/laravel.log (Last 24h)
═══════════════════════════════════════════════════

❌ CRITICAL (2 occurrences):
[2025-10-21 14:30:45] Database connection failed
[2025-10-21 15:22:10] Database connection failed

⚠️ ERROR (5 occurrences):
[2025-10-21 10:15:33] Undefined variable: user
[2025-10-21 11:42:18] Undefined variable: user
[2025-10-21 13:05:44] Undefined variable: user
[2025-10-21 16:20:11] File not found: /path/to/file
[2025-10-21 18:45:22] File not found: /path/to/file

📊 Summary:
- CRITICAL: 2
- ERROR: 5
- Total: 7 errors in last 24 hours

3. Tail Logs (Real-Time)

Live monitoring with auto-refresh:

bash .claude/skills/log-analysis-tools/log-tools.sh tail

# Features:
# - Auto-refresh every 2 seconds
# - Color-coded by level
# - Filters out INFO/DEBUG by default
# - Press 'f' to filter by pattern
# - Press 'q' to quit

Tail specific error level:

bash .claude/skills/log-analysis-tools/log-tools.sh tail --level ERROR
bash .claude/skills/log-analysis-tools/log-tools.sh tail --level WARNING

4. Search Logs

Search with regex support:

# Search for pattern
bash .claude/skills/log-analysis-tools/log-tools.sh search "database connection"

# Case-insensitive
bash .claude/skills/log-analysis-tools/log-tools.sh search -i "undefined variable"

# Regex pattern
bash .claude/skills/log-analysis-tools/log-tools.sh search "ERROR.*user"

# Show context (5 lines before/after)
bash .claude/skills/log-analysis-tools/log-tools.sh search "exception" --context 5

Output with context:

🔍 Search results: "database connection" in storage/logs/laravel.log
═════════════════════════════════════════════════════════════════════

Match 1 (Line 4523):
---
[2025-10-21 14:30:42] INFO: Attempting database connection
[2025-10-21 14:30:43] INFO: Connection timeout exceeded
[2025-10-21 14:30:44] ERROR: Database connection failed ← MATCH
[2025-10-21 14:30:45] CRITICAL: Application halted
[2025-10-21 14:30:46] INFO: Retry scheduled for 14:31:00
---

Total: 3 matches found

5. Prune Logs (Archive Old Entries)

THE KEY FEATURE: Keep today's logs, archive older

# Auto-prune: Keep today, archive rest
bash .claude/skills/log-analysis-tools/log-tools.sh prune

# Custom retention (keep last N days)
bash .claude/skills/log-analysis-tools/log-tools.sh prune --keep 7
bash .claude/skills/log-analysis-tools/log-tools.sh prune --keep 30

# Dry run (preview what will be archived)
bash .claude/skills/log-analysis-tools/log-tools.sh prune --dry-run

What it does:

📦 Log Pruning Process
═══════════════════════

Current log size: 450 MB
Target: Keep logs from 2025-10-21 (today)

Will archive:
- 2025-10-20 (15 MB) → storage/logs/archive/2025-10-20.log.gz
- 2025-10-19 (22 MB) → storage/logs/archive/2025-10-19.log.gz
- 2025-10-18 (18 MB) → storage/logs/archive/2025-10-18.log.gz
... (older entries)

After pruning:
- Active log: 12 MB (today only)
- Archived: 438 MB (compressed)
- Space saved: 73% (gzip compression)

Proceed? (Y/n)

Archive Structure:

storage/logs/
├── laravel.log              (12 MB - today only)
└── archive/
    ├── 2025-10-20.log.gz    (15 MB compressed)
    ├── 2025-10-19.log.gz    (22 MB compressed)
    ├── 2025-10-18.log.gz    (18 MB compressed)
    └── ...

Benefits:

• ✅ 99% of debugging uses today's logs only
• ✅ Massive performance improvement (12 MB vs 450 MB)
• ✅ Faster log viewing (lnav loads 38x faster)
• ✅ Old logs preserved (compressed, searchable if needed)
• ✅ Automatic cleanup prevents disk space issues

6. Statistics

Get log file insights:

bash .claude/skills/log-analysis-tools/log-tools.sh stats

# Shows:
# - File size
# - Date range
# - Error counts by level
# - Top 10 error messages
# - Errors per hour (chart)

Output Example:

📊 Log Statistics: storage/logs/laravel.log
═══════════════════════════════════════════════════

📁 File Info:
- Size: 450 MB
- Lines: 2,450,823
- Date Range: 2025-09-15 to 2025-10-21 (36 days)

📈 Error Breakdown (Last 7 days):
- CRITICAL: 12  (0.5%)
- ERROR: 245    (10.2%)
- WARNING: 567  (23.5%)
- INFO: 1,588   (65.8%)

🔥 Top 10 Errors:
1. Database connection timeout (87 times)
2. Undefined variable: user (45 times)
3. File not found: /uploads/ (34 times)
4. Memory limit exceeded (23 times)
5. API rate limit hit (18 times)
...

⏰ Errors Per Hour (Last 24h):
00:00 ▁▁
02:00 ▁
04:00 ▂
06:00 ▃
08:00 ████ (peak)
10:00 ███
12:00 ▅
14:00 ██
16:00 ████ (peak)
18:00 ▂
20:00 ▁
22:00 ▁

💡 Recommendation: Peak errors at 08:00 and 16:00 - investigate cron jobs or peak traffic patterns

7. Multi-File Analysis

Merge logs from multiple sources:

# Merge Laravel + Apache logs with timestamp sync
bash .claude/skills/log-analysis-tools/log-tools.sh merge \
    storage/logs/laravel.log \
    /var/log/apache2/error.log

# lnav automatically:
# - Merges by timestamp
# - Color-codes by source
# - Allows filtering by file

Use Case: Correlate application errors with server errors

🎬 Workflow Examples

Example 1: Debugging Production 500 Error

User: "I'm getting 500 errors in production, check logs"

Skill activates:

# 1. Auto-detect framework (Laravel)
# 2. Show errors from last 24h
bash .claude/skills/log-analysis-tools/log-tools.sh errors

# Output shows:
# [2025-10-21 16:45:22] ERROR: Undefined variable: user in PaymentController.php:45
# (12 occurrences in last hour)

# 3. Get context around first occurrence
bash .claude/skills/log-analysis-tools/log-tools.sh search "PaymentController.php:45" --context 10

# Shows stack trace and request details

Result: Error identified in 30 seconds (vs 10 minutes reading full log)

Example 2: Performance Investigation

User: "Application slow this morning, check logs"

Skill activates:

# 1. View statistics
bash .claude/skills/log-analysis-tools/log-tools.sh stats

# Shows peak errors at 08:00 AM
# Top error: "Database connection timeout" (87 times)

# 2. Search for database timeouts
bash .claude/skills/log-analysis-tools/log-tools.sh search "database.*timeout" --context 3

# Shows all timeout occurrences with context

Result: Database timeout identified as root cause

Example 3: Log Cleanup

User: "Logs are huge, clean them up"

Skill activates:

# 1. Check current size
bash .claude/skills/log-analysis-tools/log-tools.sh stats
# Shows: 450 MB log file (slow to open)

# 2. Prune (keep today only)
bash .claude/skills/log-analysis-tools/log-tools.sh prune

# Result:
# - Active log: 12 MB (38x smaller)
# - Archived: 438 MB compressed
# - lnav opens instantly instead of 15 second wait

Result: 97% reduction in active log size, massive performance boost

🔧 Installation

Required

lnav - Advanced log viewer:

# Mac
brew install lnav

# Ubuntu/Debian
apt install lnav

# Windows (WSL)
apt install lnav

Optional (Enhanced Features)

jq - JSON log parsing:

# Mac
brew install jq

# Linux
apt install jq

gzip - Log compression (usually pre-installed):

# Verify
gzip --version

📁 Framework Auto-Detection Logic

Detection priority:

1. Check for framework files (highest priority):

   # Laravel
   [ -f "artisan" ] && [ -f "composer.json" ] → Laravel
   
   # CodeIgniter 4
   [ -f "spark" ] && [ -d "app/Config" ] → CodeIgniter 4
   
   # CodeIgniter 3
   [ -f "index.php" ] && [ -d "application/config" ] → CodeIgniter 3
   
   # Symfony
   [ -f "bin/console" ] && [ -f "composer.json" ] → Symfony
   
   # Next.js
   [ -f "next.config.js" ] || [ -d ".next" ] → Next.js
   
   # Nuxt
   [ -f "nuxt.config.js" ] || [ -d ".nuxt" ] → Nuxt
   
   # Express
   [ -f "package.json" ] && grep "express" package.json → Express
   
   # Django
   [ -f "manage.py" ] → Django
   
   # Flask
   [ -f "app.py" ] && grep "Flask" app.py → Flask
   

2. Check package.json dependencies:

   # React
   grep "react" package.json → React (client-side only)
   
   # Vue
   grep "vue" package.json → Vue (client-side only)
   

3. Check for common log files:

   # Laravel
   [ -f "storage/logs/laravel.log" ] → Laravel
   
   # CI4
   [ -d "writable/logs" ] && ls writable/logs/*.log → CodeIgniter 4
   
   # CI3
   [ -d "application/logs" ] → CodeIgniter 3
   

4. Default fallback:

   • Ask user: "Which framework is this project using?"
   • Show detected files for confirmation

🎯 Token Efficiency

Traditional Approach (Reading Entire Log):

User: "Find errors in laravel.log"
Claude reads entire 450 MB file → 180,000 tokens
Claude processes → finds errors
Total: ~180,000 tokens

With log-analysis-tools Skill:

User: "Find errors in laravel.log"
Skill runs: bash log-tools.sh errors
Returns: 50 error lines (200 tokens)
Total: ~400 tokens
Savings: 99.8%

Daily Operations (10 log checks):

• Traditional: 1.8M tokens/day
• With skill: 4,000 tokens/day
• Savings: 99.8% = 1,796,000 tokens/day

Monthly Savings:

• Tokens: 53.8M tokens saved
• Cost: $161/month saved (at $3/million tokens)
• Time: 50 hours saved (instant vs manual reading)

🔄 Integration with log-analyzer Agent

This skill provides tools, the agent provides intelligence:

User: "Analyze production logs for critical issues"

→ log-analyzer AGENT activates:
1. Uses log-analysis-tools SKILL to extract errors
2. Analyzes error patterns (frequency, timing, impact)
3. Identifies root causes
4. Provides prioritized recommendations
5. Suggests fixes for top 3 critical issues

Result: Intelligent analysis + actionable recommendations

See: agents/log-analyzer/ for the intelligent agent that uses this skill

🐛 Troubleshooting

"lnav: command not found"

Solution: Install lnav using package manager for your OS

"Permission denied" when reading logs

Solution: Run with appropriate permissions or adjust log file permissions

# Make log readable
chmod 644 storage/logs/laravel.log

# Or run with sudo (not recommended for regular use)
sudo bash log-tools.sh view

"Cannot detect framework"

Solution: Manually specify framework:

bash log-tools.sh view laravel
bash log-tools.sh view codeigniter

"Log file too large, lnav is slow"

Solution: Prune logs first!

bash log-tools.sh prune
# Then open with lnav

📝 Summary

This skill provides:

• ✅ Multi-framework support - Laravel, CodeIgniter, React, Next.js, Symfony, Django, Flask, Express
• ✅ Auto-detection - Identifies framework and log locations automatically
• ✅ Log pruning - Keep today's logs, archive older (99% use case covered)
• ✅ Fast error extraction - Find errors in seconds, not minutes
• ✅ Real-time monitoring - Live tail with filtering
• ✅ Token efficiency - 99.8% token savings vs reading entire logs
• ✅ Multi-file correlation - Merge application + server logs
• ✅ Context-aware search - See what happened before/after errors
• ✅ Statistics & insights - Error patterns, peak times, top issues

Use this skill for all log analysis operations in development workflows!

Version History

v1.0.0 (2025-10-21)

• Initial release
• Multi-framework support (Laravel, CodeIgniter, React, Next.js, Symfony, Django, Flask, Express)
• Auto-detection of framework and log locations
• Log pruning capability (keep today, archive older)
• Error extraction and analysis
• Real-time tail monitoring
• Search with context
• Statistics and insights
• 99.8% token savings vs traditional approach