Secret Scanning

Secret Scanning

Overview

Detect hardcoded secrets — API keys, tokens, passwords, PEM private keys, AWS access keys, GitHub PATs, JWTs, Slack tokens, OpenAI keys — before they land in commits, pushes, or production. This skill wraps three layered tools:

The recommended posture in 2026 is layered: gitleaks at pre-commit for speed, trufflehog with --only-verified in CI to eliminate triage burden, in-house regex as a backstop.

Companion skills:

• dep-currency-check — dependency CVE check; closest local pattern for the gate-wrapper architecture
• python-auth-security — credential storage patterns (where the secrets SHOULD live)
• docker-security — secrets-in-images (separate domain)
• llm-security — secrets in LLM prompts (don't put secrets in prompts; LLM07 / LLM06)

0. Secrets storage standard (host-wide, adopted 2026-07-25)

Detection is half the job. Where secrets LIVE is the other half — and the reason detection keeps having work to do. The standard:

~/.secrets/<project>.env   ->   loader   ->   process environment   ->   your code
   storage (0600)                              delivery (ephemeral)

Storage and delivery are different questions. An environment variable is how a process receives a secret, not where one is kept. Full spec, rationale, honest caveats and the migration recipe: references/storage-standard.md.

Enforcement — one pattern set, three call sites

The commit hook, the push hook and the gate all invoke the SAME scanner, so their rules cannot drift apart:

bash ~/.claude/skills/secret-scanning/hooks/install.sh   # per repo, commit-time

Two deliberate design choices, both learned the hard way:

• install.sh REFUSES to install if no scanner resolves. An inert hook is worse than no hook — it advertises protection that does not exist.
• The hook fails CLOSED if the scanner disappears after install. Same reasoning. git commit --no-verify remains the explicit, visible escape hatch.

Commit-time is the one that matters most: push-time stops a secret leaving the machine, commit-time stops it entering history at all.

Honest limits — do not oversell

• ~/.secrets/ is plaintext on disk. Fewer copies and one audit point, but it is NOT encryption and NOT a secret manager. Upgrade path: age+sops or pass (none installed on this host today).
• Env vars are not a security boundary — readable via /proc/<pid>/environ by the same user, inherited by every subprocess, visible in ps. Fine for delivery.
• Scrubbing is not rotation. A credential that was ever committed must be rotated at the provider; removing it from files does nothing for the exposure.

1. Tool selection — when to use which

Why three tools instead of one: different tools catch different patterns. gitleaks is regex-heavy and fast; trufflehog adds live-verification (the secret is actually valid against its provider's API) which drastically reduces false-positives. The in-house catalog covers patterns specific to our local ecosystem (e.g. .bob-checkpoint.md token-shaped strings that aren't real but pattern-match) and runs offline.

2. Install Commands

RHEL 9 / AlmaLinux 9 / Rocky 9

# gitleaks — official binary release
curl -sSfL https://github.com/gitleaks/gitleaks/releases/latest/download/gitleaks_linux_x64.tar.gz | sudo tar -C /usr/local/bin -xzf - gitleaks
sudo chmod +x /usr/local/bin/gitleaks
gitleaks version

# trufflehog — official binary release
curl -sSfL https://raw.githubusercontent.com/trufflesecurity/trufflehog/main/scripts/install.sh | sudo sh -s -- -b /usr/local/bin
trufflehog --version

Debian 12 / Ubuntu 24.04

# gitleaks
curl -sSfL https://github.com/gitleaks/gitleaks/releases/latest/download/gitleaks_linux_x64.tar.gz | sudo tar -C /usr/local/bin -xzf - gitleaks
sudo chmod +x /usr/local/bin/gitleaks

# trufflehog
curl -sSfL https://raw.githubusercontent.com/trufflesecurity/trufflehog/main/scripts/install.sh | sudo sh -s -- -b /usr/local/bin

macOS

brew install gitleaks trufflesecurity/trufflehog/trufflehog

Windows 11

# gitleaks — via Scoop / Chocolatey or release ZIP
scoop install gitleaks
# OR
choco install gitleaks

# trufflehog — via release binary; no Scoop package in 2026
# Download from https://github.com/trufflesecurity/trufflehog/releases/latest

Verify availability via env-adoption probe:

bash ~/.claude/skills/env-adoption/scripts/probe.sh check --force
jq '.tools | {gitleaks, trufflehog}' ~/.claude/state/inventory.json

3. Canonical patterns

3.1 Local pre-push hook (bash for POSIX shells, Python for cross-platform)

Two installer/scanner pairs ship in your project root's scripts/ — pick by environment:

• POSIX shells (Linux / macOS / Git Bash): scripts/install-pre-push-hook.sh installs a hook that runs secrets-scan.sh.
• Cross-platform / enterprise Windows (Execution Policy / AppLocker / Constrained Language Mode blocks PowerShell, but Python is present): scripts/install-pre-push-hook.py wires scripts/secrets-scan.py — stdlib-only, full parity with the bash pair (same patterns, allowlist, severity tiers, exit codes). Idempotent; backs up any unmanaged pre-push hook to pre-push.bak; --uninstall removes only the managed hook.

# Python pair (Linux, macOS, Windows)
python3 scripts/install-pre-push-hook.py                      # install into cwd
python3 scripts/install-pre-push-hook.py --target-repo PATH   # install into a specific repo
python3 scripts/install-pre-push-hook.py --uninstall          # remove the managed hook

To add gitleaks as a second pre-push pass, append to .git/hooks/pre-push (after the existing inhouse scan):

# gitleaks pre-push gate (additional layer)
if command -v gitleaks >/dev/null 2>&1; then
  gitleaks dir . --no-banner --redact --exit-code 1 >&2 || {
    echo "[gitleaks] BLOCKING — secrets detected. Override: git push --no-verify"
    exit 1
  }
fi

3.2 GitHub Actions CI workflow

# .github/workflows/secret-scan.yml
name: secret-scan
on: [pull_request, push]
jobs:
  gitleaks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - uses: gitleaks/gitleaks-action@v2
        env:
          GITLEAKS_NOTIFY_USER_LIST: '@security-team'

  trufflehog-verified:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - name: trufflehog (verified only)
        uses: trufflesecurity/trufflehog@main
        with:
          extra_args: --only-verified --fail

3.3 G_SECRETS_SCAN gate (S038 Batch B)

The gate at _meta/gates.py wraps all three scanners. Auto-picks gitleaks > trufflehog > in-house.

# Advisory mode (never fails build; reports findings)
python3 ~/.claude/skills/_meta/gates.py G_SECRETS_SCAN /path/to/project \
  --secrets-mode advisory

# Strict mode (fails on any finding)
python3 ~/.claude/skills/_meta/gates.py G_SECRETS_SCAN /path/to/project \
  --secrets-mode strict --scanner gitleaks

Exit codes match the existing gate contract:

• 0 = pass (no findings OR advisory mode)
• 2 = fail (strict mode AND findings detected)
• 3 = environmental error (no scanner available)

3.4 gitleaks custom config (.gitleaks.toml at repo root)

The dep-currency-check skill has an established allowlist convention. Mirror that pattern for project-specific noise:

# .gitleaks.toml
title = "<project>-gitleaks-config"

[extend]
useDefault = true

[allowlist]
description = "project-specific known FPs"
paths = [
  '''docs/.*tutorial.*\.md''',
  '''.*\.test\.(js|ts|py)$''',
]
regexes = [
  '''aws_access_key_id\s*=\s*AKIAIOSFODNN7EXAMPLE''',  # AWS docs example
  '''ghp_[A-Za-z0-9]{36}.*example.*''',                 # placeholder PATs
]

4. Defense-in-depth layering

Use ALL THREE tools — they catch different things:

Trufflehog's --only-verified flag is the headline 2026 improvement — by hitting each candidate against its real provider API, it eliminates almost all false-positives. The trade-off is speed (each verify is an HTTP round-trip) and the assumption that you trust your CI runner to make those API calls (in air-gapped environments, skip --only-verified).

5. Security Hardening

1. .gitignore .env* and *.pem before adding the scan — the scan is the safety net, not the policy.
2. Pre-commit AND pre-push, not just one — pre-commit catches local mistakes; pre-push catches anything that bypassed it (rebase, cherry-pick, force-push to a new branch).
3. CI gate on PR + push — local hooks can be bypassed with --no-verify; CI cannot.
4. Custom config per project — every project accumulates legitimate FP patterns over time. Maintain a .gitleaks.toml (or equivalent) with documented justifications.
5. Rotate on detection, don't just delete — if a real secret was committed, even briefly, treat it as leaked. Rotate the secret with the provider. git filter-repo to scrub history is theatre if the secret was visible for any minute via cloning/pushing.
6. Audit log scanner runs — when, who, what findings. The G_SECRETS_SCAN gate emits stdout; capture it in CI artifacts.
7. Don't ignore --no-verify overrides — log who used the override and investigate. If --no-verify is routine in your team, the hook is too noisy and needs allowlist tuning.
8. Tune the in-house allowlist with care — every entry is a documented exception. Review the allowlist quarterly; remove entries whose context no longer applies.
9. Scan generated artifacts too — build outputs, Docker images (trivy fs overlaps here), test fixtures with mock secrets. A "test-only" secret committed by accident is still a secret if it works against any real service.
10. Keep scanner versions current via dep-currency-check — gitleaks and trufflehog ship pattern updates regularly. A stale scanner misses new providers' secret formats.

6. Anti-patterns

7. Selection Cheatsheet

• Need a pre-commit hook right now → gitleaks protect --staged
• Need a CI gate with zero false positives → trufflehog --only-verified --fail
• Need a one-shot scan of an existing repo's full history → trufflehog git file://. --no-update
• Need a defense-in-depth backstop offline → in-house scripts/secrets-scan.sh (always present in this ecosystem)
• Need a programmatic gate inside bob WP-completion → gates.py G_SECRETS_SCAN <root> --secrets-mode strict
• Need a forge Step 1 advisory check → gates.py G_SECRETS_SCAN <root> --secrets-mode advisory

8. Gotchas

9. Update triggers (alf scans these)

• gitleaks major version bump (currently 8.x as of 2026-05)
• trufflehog major version bump (currently 3.x)
• New provider added to gitleaks default config (auto-detected via gitleaks version output)
• New secret-shape provider widely deployed (e.g. a new cloud's tokens become common in the wild)
• In-house scripts/secrets-scan.sh pattern catalog update
• Annual review on 2027-05-24

10. See Also