1746 Audit Logging 74e8a40e

Audit Logging

MCP Gateway Registry provides comprehensive audit logging for compliance, security monitoring, and operational visibility. All API requests and MCP server access events are logged to MongoDB/DocumentDB with automatic retention management.

Overview

Audit logging captures two types of events:

1. Registry API Access - All REST API requests to the Registry (/api/*, /v0.1/*)
2. MCP Server Access - All MCP protocol requests proxied through the Gateway

Sensitive data such as authentication tokens, session cookies, and passwords are never logged. Credentials are masked to show only the last 6 characters as a hint for debugging.

Security and Privacy

Data That Is NOT Logged

The following sensitive data is explicitly excluded from audit logs:

• Authentication tokens (Bearer tokens, JWT tokens)
• Session cookies (Cookie header values)
• Passwords (form fields, query parameters)
• API keys (full values)
• Refresh tokens
• Authorization header values

Data Masking

When credential hints are logged for debugging purposes, they are automatically masked:

• Full token: eyJhbGciOiJSUzI1NiIsInR5... becomes ***zI1Ni
• Tokens shorter than 6 characters become ***

Query parameters with sensitive names (token, password, key, secret, api_key, etc.) are automatically masked.

Event Schemas

Registry API Access Event

Logged for every REST API request to the Registry.

{
  "timestamp": "2026-02-06T10:30:00.000Z",
  "log_type": "registry_api_access",
  "version": "1.0",
  "request_id": "abc123-def456-...",
  "correlation_id": null,
  "identity": {
    "username": "john.doe@example.com",
    "auth_method": "oauth2",
    "provider": "keycloak",
    "groups": ["mcp-registry-admin", "developers"],
    "scopes": ["registry-admins"],
    "is_admin": true,
    "credential_type": "session_cookie",
    "credential_hint": "***abc123"
  },
  "request": {
    "method": "POST",
    "path": "/api/servers",
    "query_params": {},
    "client_ip": "192.168.1.100",
    "forwarded_for": "10.0.0.1",
    "user_agent": "Mozilla/5.0...",
    "content_length": 1024
  },
  "response": {
    "status_code": 201,
    "duration_ms": 45.32,
    "content_length": 512
  },
  "action": {
    "operation": "create",
    "resource_type": "server",
    "resource_id": "my-mcp-server",
    "description": "Create new MCP server"
  },
  "authorization": {
    "decision": "ALLOW",
    "required_permission": "servers:write",
    "evaluated_scopes": ["registry-admins"]
  }
}

MCP Server Access Event

Logged for every MCP protocol request proxied through the Gateway.

{
  "timestamp": "2026-02-06T10:30:00.000Z",
  "log_type": "mcp_server_access",
  "version": "1.0",
  "request_id": "xyz789-...",
  "correlation_id": null,
  "identity": {
    "username": "ai-agent@example.com",
    "auth_method": "jwt_bearer",
    "provider": "keycloak",
    "groups": [],
    "scopes": ["mcp-server-cloudflare-docs"],
    "is_admin": false,
    "credential_type": "bearer_token",
    "credential_hint": "***def456"
  },
  "mcp_server": {
    "name": "cloudflare-docs",
    "path": "/cloudflare-docs",
    "version": "1.0.0",
    "proxy_target": "http://internal-mcp-server:8080/mcp"
  },
  "mcp_request": {
    "method": "tools/call",
    "tool_name": "search_docs",
    "resource_uri": null,
    "mcp_session_id": "session-123",
    "transport": "streamable-http",
    "jsonrpc_id": "1"
  },
  "mcp_response": {
    "status": "success",
    "duration_ms": 123.45,
    "error_code": null,
    "error_message": null
  }
}

Data Fields Reference

Identity Fields

Action Fields (Registry API only)

MCP Request Fields (MCP Access only)

Data Retention

Audit logs are automatically expired using MongoDB/DocumentDB TTL (Time-To-Live) indexes.

Default Retention

• Default retention period: 7 days
• TTL index field: timestamp

Configuring Retention

Set the AUDIT_LOG_MONGODB_TTL_DAYS environment variable to customize retention:

# Keep logs for 30 days
export AUDIT_LOG_MONGODB_TTL_DAYS=30

# Keep logs for 90 days (compliance requirement)
export AUDIT_LOG_MONGODB_TTL_DAYS=90

The TTL index is created when running the DocumentDB initialization script:

./scripts/init-documentdb.sh

Important Notes

• TTL indexes run approximately once per minute in MongoDB/DocumentDB
• Documents may persist slightly longer than the TTL value
• Changing the TTL requires dropping and recreating the index with --recreate flag
• For compliance requirements, consider also streaming logs to a long-term archive

Storage

MongoDB Collection

Audit events are stored in the audit_events_{namespace} collection with the following indexes:

Storage Sizing

Typical event sizes:

• Registry API event: ~1-2 KB
• MCP Server Access event: ~1-2 KB

Estimated storage (without compression):

• 1,000 requests/day for 7 days: ~14 MB
• 10,000 requests/day for 30 days: ~600 MB
• 100,000 requests/day for 90 days: ~18 GB

Viewing Audit Logs

Admin UI

Administrators can view audit logs in the Registry UI:

1. Navigate to Settings > Audit > Audit Logs
2. Select log stream: Registry API or MCP Access
3. Apply filters (time range, username, operation, status)
4. Click any row to view full event details
5. Export filtered results as JSONL or CSV

API Access

Query audit events programmatically:

# Get recent Registry API events
curl -H "Authorization: Bearer $TOKEN" \
  "https://registry.example.com/api/audit/events?stream=registry_api&limit=50"

# Get MCP access events for a specific user
curl -H "Authorization: Bearer $TOKEN" \
  "https://registry.example.com/api/audit/events?stream=mcp_access&username=john.doe"

# Export events as JSONL
curl -H "Authorization: Bearer $TOKEN" \
  "https://registry.example.com/api/audit/export?stream=registry_api&format=jsonl"

MongoDB/DocumentDB Direct Query

// Find all events for a user in the last 24 hours
db.audit_events_default.find({
  "identity.username": "john.doe@example.com",
  "timestamp": { $gte: new Date(Date.now() - 24*60*60*1000) }
}).sort({ timestamp: -1 })

// Count events by operation type
db.audit_events_default.aggregate([
  { $match: { log_type: "registry_api_access" } },
  { $group: { _id: "$action.operation", count: { $sum: 1 } } },
  { $sort: { count: -1 } }
])

// Find failed MCP requests
db.audit_events_default.find({
  "log_type": "mcp_server_access",
  "mcp_response.status": "error"
})

Configuration

Environment Variables

Non-Blocking Design

Audit logging is designed to never impact request processing:

• Logging happens asynchronously after the response is sent
• Failures in audit logging are logged as warnings but don't fail requests
• High-volume scenarios use batched writes (if enabled)

Compliance Considerations

SOC 2 / ISO 27001

Audit logs support compliance requirements by capturing:

• Who: User identity with auth method and provider
• What: Operation performed with resource details
• When: Precise UTC timestamp
• Where: Client IP and forwarded-for headers
• Outcome: Success/failure status with error details

GDPR

• User identifiers (usernames) are logged for accountability
• No PII beyond usernames is captured
• Logs can be exported and deleted per data subject requests
• TTL-based retention supports data minimization

Additional Recommendations

For production compliance deployments:

1. Stream audit logs to a SIEM (Splunk, Datadog, etc.) for long-term retention
2. Set up alerts for suspicious patterns (failed auths, privilege escalation)
3. Regularly review admin actions in the audit log
4. Document your retention policy and ensure TTL matches it

Troubleshooting

Logs Not Appearing

1. Verify audit logging is enabled: AUDIT_LOG_ENABLED=true
2. Check MongoDB connection: ensure the Registry can write to the database
3. Look for warnings in Registry logs: grep "audit" registry.log

TTL Not Working

1. Verify the TTL index exists: db.audit_events_default.getIndexes()
2. Note that MongoDB TTL runs approximately every 60 seconds
3. Documents may persist up to 60 seconds beyond their expiration time

Missing Events

1. Check if the request completed (cancelled requests may not be logged)
2. Verify the log stream filter matches the event type
3. For MCP access, ensure the path is not an API path (starts with /api/)