Nginx Request Logging

Nginx Request Logging Configuration

This skill provides guidance for configuring Nginx web servers with custom logging, rate limiting, and error handling.

When to Use This Skill

Apply this skill when tasks involve:

• Installing and configuring Nginx
• Setting up custom log formats
• Implementing rate limiting
• Creating custom error pages (404, 500, etc.)
• Configuring Nginx to listen on non-standard ports

Pre-Configuration Analysis

Before modifying any Nginx configuration:

1. Examine existing configuration structure

   • Read /etc/nginx/nginx.conf to understand the current setup
   • Check for existing include directives to understand file organization
   • Identify where log formats, rate limiting zones, and other global settings are defined

2. Check system state

   • Verify if Nginx is already installed: which nginx or nginx -v
   • Check if Nginx is already running: pgrep nginx or ps aux | grep nginx
   • Verify if the target port is available: ss -tlnp | grep <port> or netstat -tlnp | grep <port>

3. Backup original configuration

   • Create a backup before modifications: cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak

Configuration Approach

Directory Structure

Nginx configurations typically follow this hierarchy:

• /etc/nginx/nginx.conf - Main configuration (global settings, log formats, rate limiting zones)
• /etc/nginx/conf.d/ - Site-specific configurations (server blocks)
• /etc/nginx/sites-available/ and /etc/nginx/sites-enabled/ - Alternative site management (Debian-based)

Configuration Placement Guidelines

Rate Limiting Configuration

Rate limiting requires two parts:

1. Zone definition (in http block of nginx.conf):

   limit_req_zone $binary_remote_addr zone=zonename:10m rate=10r/s;
   

2. Zone application (in server or location block):

   limit_req zone=zonename burst=5 nodelay;
   

Custom Log Format

Define custom log formats in the http block:

log_format custom_format '$remote_addr - $remote_user [$time_local] '
                         '"$request" $status $body_bytes_sent '
                         '"$http_referer" "$http_user_agent"';

Apply in server block:

access_log /var/log/nginx/custom_access.log custom_format;

Service Management

Nginx service management varies by environment:

Important: Always test configuration before starting/reloading:

nginx -t

Verification Strategies

Basic Functionality

curl -s http://localhost:<port>/
curl -s -o /dev/null -w "%{http_code}" http://localhost:<port>/nonexistent

Rate Limiting Verification

Rate limiting requires concurrent requests to trigger. Sequential requests will not exceed the rate limit.

Correct approach (parallel requests):

seq 20 | xargs -P 20 -I {} curl -s -o /dev/null -w "%{http_code}\n" http://localhost:<port>/

Incorrect approach (will not trigger rate limiting):

for i in {1..20}; do curl -s http://localhost:<port>/; done  # Too slow, sequential

Log Verification

tail -f /var/log/nginx/access.log
tail -f /var/log/nginx/error.log

Common Pitfalls

1. Log format not found: Log format must be defined in nginx.conf before being referenced in server blocks

2. Rate limiting not triggering: Sequential requests are too slow; use parallel requests with xargs -P or similar

3. Configuration syntax errors: Always run nginx -t before starting or reloading

4. Port already in use: Check with ss -tlnp before configuring a new port

5. systemctl not available: In containers or minimal environments, use nginx command directly

6. Default site conflicts: Remove or disable default site configuration when creating custom configurations:

   rm -f /etc/nginx/sites-enabled/default
   

7. Missing directories: Verify required directories exist before writing configuration:

   ls -la /etc/nginx/conf.d/
   

Execution Efficiency

• Batch file operations: Create multiple static files (index.html, 404.html, etc.) in parallel when possible
• Combine verification steps: Test multiple endpoints in a single verification pass
• Plan verification upfront: Determine the testing strategy before implementation
• Use idempotent commands: Prefer mkdir -p, rm -f to handle existing/missing files gracefully

Example Workflow

1. Check system state (Nginx installed, running, port availability)
2. Read existing nginx.conf structure
3. Backup configuration
4. Create required directories and static content
5. Modify nginx.conf for global settings (log format, rate limiting zone)
6. Create server configuration in conf.d/
7. Remove conflicting default configurations
8. Test configuration with nginx -t
9. Start/reload Nginx service
10. Verify all functionality (main page, error pages, rate limiting, logs)