Secret Scanning and Management

Secret Scanning and Management

Overview

This skill focuses on detecting, managing, and preventing the leakage of sensitive information such as API keys, passwords, tokens, and certificates within the codebase and version control history.

Core Principles

1. No Hardcoded Secrets

Source code should never contain sensitive data.

• Separation of code and config: Store configuration (including secrets) in the environment, separate from the application code (as per 12-Factor App principles).
• Environment Variables: Use .env files for local development but ensure they are strictly ignored in version control (.gitignore).

2. Secret Scanning

Actively scan the repository for accidentally committed secrets.

• CI/CD Integration: Integrate tools like TruffleHog, Gitleaks, or GitHub Advanced Security into your CI pipeline to scan commits before they are merged.
• Historical Scanning: Scan the entire git history, not just the current state of the code, because deleted secrets in previous commits are still compromised.

3. Pre-commit Hooks

Prevent secrets from ever entering the git history.

• Client-side checks: Set up tools like pre-commit to run secret scanners locally before a developer can successfully commit code.

4. Secret Managers

Use dedicated systems to store and inject secrets at runtime.

• Centralized Management: Use solutions like AWS Secrets Manager, HashiCorp Vault, Azure Key Vault, or Google Secret Manager.
• Dynamic Secrets: Where possible, prefer short-lived, dynamically generated credentials over long-lived static keys.

5. Remediation

If a secret is leaked, it must be considered compromised immediately.

• Revoke and Rotate: Never just delete the secret from the code. You must revoke the leaked key at the provider and generate a new one.

Code Examples

Setting up a Pre-commit Hook for Gitleaks

Create a .pre-commit-config.yaml in your repository root:

repos:
  - repo: https://github.com/zricethezav/gitleaks
    rev: v8.18.1
    hooks:
      - id: gitleaks

Regular Expressions for Secret Detection

Example regex pattern to catch AWS Access Keys:

(?i)aws_access_key_id\s*=\s*(?P<key>AKIA[0-9A-Z]{16})

Checklist for Secret Management

• Are all .env files and configuration files containing secrets added to .gitignore?
• Is a secret scanner (e.g., Gitleaks) configured in the CI/CD pipeline?
• Are pre-commit hooks configured to prevent developers from committing secrets locally?
• Are secrets retrieved from a secure Secret Manager in production rather than plain environment variables?
• Is there an established procedure to rotate secrets immediately if they are leaked?