Secrets Rotation Patterns

Secrets Rotation Patterns

Overview

Secrets rotation ensures credentials are regularly updated to minimize the impact of credential compromise. AWS provides native rotation for some services (RDS, Redshift, DocumentDB) and supports custom rotation for others.

sequenceDiagram
    participant SM as Secrets Manager
    participant Lambda as Rotation Lambda
    participant DB as Database
    participant App as Application
    
    SM->>Lambda: Trigger rotation
    Lambda->>SM: Create new secret version (AWSPENDING)
    Lambda->>DB: Set new password
    Lambda->>SM: Test new credentials
    Lambda->>SM: Promote AWSPENDING to AWSCURRENT
    App->>SM: Get secret (returns new value)

Key Concepts

Rotation Phases

1. createSecret - Generate new secret value
2. setSecret - Apply to target resource
3. testSecret - Verify new credentials work
4. finishSecret - Promote new version to current

Rotation Strategies

Best Practices

1. Use managed rotation when available (RDS, etc.)
2. Short rotation periods - 30 days or less for sensitive credentials
3. Test rotation in dev first - ensure applications handle credential refresh
4. Multi-user rotation - use alternating users to avoid downtime
5. Monitor rotation failures - alert on failed rotations
6. Cache secrets - reduce API calls with TTL-based caching
7. Use VPC endpoints - access Secrets Manager without internet

Anti-Patterns to Avoid

❌ Hardcoded secrets in code or config files
❌ Long-lived credentials without rotation
❌ Sharing secrets across environments
❌ No monitoring for rotation failures
❌ Storing secrets in environment variables without encryption

Example 1: Terraform - RDS Credentials Rotation

This example creates:

• RDS instance with master credentials in Secrets Manager
• Automatic rotation schedule (every 30 days)
• Application role with read-only secret access

📁 Location: terraform/examples/secrets-rotation/

Key Features

# Create secret with rotation
resource "aws_secretsmanager_secret" "db_credentials" {
  name        = "${local.name_prefix}/db/master"
  description = "RDS master credentials with automatic rotation"
}

# Enable rotation with native RDS rotator
resource "aws_secretsmanager_secret_rotation" "db_credentials" {
  secret_id           = aws_secretsmanager_secret.db_credentials.id
  rotation_lambda_arn = aws_lambda_function.rotator.arn

  rotation_rules {
    automatically_after_days = 30
    schedule_expression      = "rate(30 days)"
  }
}

# RDS instance using the secret
resource "aws_db_instance" "main" {
  manage_master_user_password = true  # Native rotation
  master_user_secret_kms_key_id = aws_kms_key.rds.arn
}

Example 2: CDK - API Key Rotation with Custom Lambda

This example creates:

• API key secret with custom rotation Lambda
• SSM Parameter Store for non-secret configuration
• Application integration with automatic refresh

📁 Location: cdk/examples/secrets-rotation/

Key Features

// Create secret with rotation schedule
const apiKeySecret = new secretsmanager.Secret(this, 'ApiKeySecret', {
  secretName: `${props.projectName}/${props.environment}/api-key`,
  description: 'External API key with automatic rotation',
  generateSecretString: {
    secretStringTemplate: JSON.stringify({ service: 'external-api' }),
    generateStringKey: 'apiKey',
    excludePunctuation: true,
  },
});

// Add rotation with custom Lambda
apiKeySecret.addRotationSchedule('Rotation', {
  rotationLambda: rotatorFunction,
  automaticallyAfter: Duration.days(30),
});

// Application can refresh secrets
const cachedSecret = new secretsmanager.Secret.fromSecretNameV2(
  this, 'CachedSecret', apiKeySecret.secretName
);

SSM Parameter Store vs Secrets Manager

Validation Checklist

• All database credentials in Secrets Manager (not hardcoded)
• Rotation enabled for all secrets
• Rotation schedule ≤ 30 days
• CloudWatch alarms for rotation failures
• Applications handle credential refresh
• VPC endpoint for Secrets Manager
• KMS encryption enabled

Related Skills

• IAM Least Privilege - Scoped secret access
• Secrets Externalization - Never hardcode
• Cost Governance - SSM vs Secrets Manager costs