CLI Audit Wizard

Optimization: This skill uses on-demand loading. Heavy content lives in references/ and is loaded only when needed.

Language rule: Skill instructions are written in English. When generating user-facing output, detect the project's primary language (from README, comments, docs, commit messages) and produce the report in that language. If the project is bilingual, ask the user which language to use before proceeding.

Audit Wizard -- Wizard Quality Index (WQI)

Audit configuration wizards for UX quality, lifecycle completeness, and multi-surface exposure (CLI, MCP, web).

"The worst time for users to make choices is before they've started using the system." -- OpenBSD design philosophy

Natural laws mapped to wizard rules

Each analogy maps to a concrete, testable audit check. Only actionable rules are listed here.

The full 27 analogies across 5 domains (biology, physics, information theory, game design, cognitive science) are in references/analogies.md. Each includes the biological/physical mechanism, mapping table, and a named Rule.

The 4 Laws of Good Wizards

These are non-negotiable. A wizard that violates any of these is broken. The Laws are factored into ../shared/cli-ergonomics.md so cli-audit-shell (S9), cli-forge-chef (prompt design), and cli-forge-infra (operator wrappers) can reuse the same semantics across CLI, TUI, agent prompt, and CI surfaces.

A wizard lives at rung 1 of ../shared/escalation-ladder.md — a setup CLI that asks once, recaps, and writes config-as-code. A "wizard" that drives a vendor GUI through screenshots and clicks is rung 5 and fails all 4 Laws by construction (no scripting, no recap, no replayability). If the only path is GUI driving, that's not a wizard — that's a last-resort bridge, and the lower rungs of the ladder must be exhausted first.

Law 1 -- Ask once, derive the rest (Germination)

If you can calculate a value from another, do not ask the question. step ca init asks the domain name, derives the ACME URL automatically. A good wizard asks 3-5 seed questions and grows the full config.

Law 2 -- Defaults are decisions (OpenBSD philosophy)

Every default is an opinionated choice, documented and justified. No empty strings, no null, no "please configure". Caddy defaults http_port: 80 -- you don't think about it.

Law 3 -- Recap before apply (Dry-run)

Show what will change before touching the system. OpenBSD recaps before install. Talos has --dry-run. The user must be able to review and abort.

Law 4 -- Config-as-code output (Committable)

The wizard generates a file on disk, not hidden state. The file is committable, diffable, readable. Re-running the wizard reads the existing config and proposes a diff. Never a black box.

The Lifecycle State Machine

A wizard is not a one-shot setup. It's a config lifecycle manager.

wizard --run
  |
  +-- config ABSENT -> SETUP mode
  |     +-- seed questions (3-5 max)
  |     +-- credential discovery cascade (see below)
  |     +-- recap -> confirm -> write config.toml
  |     +-- doctor run (immediate post-setup validation)
  |
  +-- config PRESENT -> DOCTOR + EDIT mode
  |     +-- lint config (missing values, deprecated keys, schema drift)
  |     +-- test connectivity (ping backends, check creds validity)
  |     +-- compare against current schema (migration needed?)
  |     +-- show current config summary
  |     +-- PROPOSE EDIT: "Modify config? [backends / credentials / rules / all / skip]"
  |     |     +-- user picks section -> wizard opens ONLY that section pre-filled
  |     |     +-- user changes values -> recap diff -> confirm -> apply
  |     |     +-- skip -> exit (config unchanged)
  |     +-- re-run credential discovery if any creds are expired/invalid
  |
  +-- explicit flags
        +-- --setup     force setup from scratch (ignores existing config)
        +-- --doctor    force doctor only (no edit prompt)
        +-- --edit [section]  jump directly to editing a section
        +-- --dry-run   show what would change, don't apply

Key behavior: re-run always reads existing config. The wizard is never amnesic. On re-run it loads current values, shows them, and lets the user modify. It never starts from zero unless --setup is explicit.

Reference implementations: rustup check, brew doctor, helm lint, talosctl health.

Credential Discovery Cascade

Credentials (API keys, OAuth tokens) are the hardest part of any setup. The wizard should never just ask "paste your API key" as first option. Instead, it runs a discovery cascade -- try the easiest path first, fall back progressively.

This is the universal three-tier hierarchy used by gh, gcloud, aws, railway, vercel, wrangler, and every well-designed CLI. Deviation is an anti-pattern.

Credential hygiene (file permissions, .env in .gitignore, raw secrets in config) is checked by cli-audit-code dimension C9 (Security & Input Validation).

credential_discovery(service):
  |
  +-- 0. ENV VAR CHECK (before any prompts -- blocks wizard if found)
  |     +-- $ANTHROPIC_API_KEY, $OPENAI_API_KEY, $MISTRAL_API_KEY, etc.
  |     +-- Found + valid? -> use it, skip interactive auth entirely
  |     +-- Found + invalid? -> warn "ANTHROPIC_API_KEY set but invalid", continue
  |     (gh pattern: "The value of GH_TOKEN is being used for authentication.")
  |
  +-- 1. SCAN -- look for existing tokens on disk
  |     +-- ~/.config/claude/credentials.json     (Claude Code)
  |     +-- ~/.config/github-copilot/hosts.json   (Copilot)
  |     +-- keychain / secret-tool / pass          (OS keyring)
  |     +-- ~/.netrc                                (legacy but common)
  |     +-- 1Password CLI / Bitwarden CLI          (if installed)
  |
  |     Found? -> "Found Claude token at ~/.config/claude/. Use it? [Y/n]"
  |               Validate token (API ping) before accepting.
  |
  +-- 2. OAUTH / DEVICE FLOW (RFC 8628 -- modern standard)
  |     +-- Show device code: "First copy your one-time code: 7CDF-8959"
  |     +-- Open browser automatically to authorization page
  |     +-- Poll until user authorizes -> receive token
  |     +-- HEADLESS FALLBACK 1: show URL + code for manual entry
  |     |     "Visit: https://provider.com/device -- Enter code: 7CDF-8959"
  |     +-- HEADLESS FALLBACK 2 (gcloud pattern): output bootstrap command
  |     |     "Run on a machine with a browser: tool auth --remote-bootstrap=..."
  |     +-- Token stored in OS keychain; plaintext fallback with warning
  |
  |     Worked? -> "Authenticated via OAuth. Token stored."
  |
  +-- 3. API KEY -- manual paste (last resort)
  |     +-- Show link to token page: "Generate a key at https://console.anthropic.com/keys"
  |     +-- "Paste your API key: ********"
  |     +-- Validate immediately (API ping)
  |     +-- Invalid? -> show expected format + "Try again or Ctrl-C to skip"
  |     +-- Valid? -> store in config (env var reference, not raw key)
  |
  +-- 4. SKIP -- defer credential setup
        +-- "No credentials configured. Some features will be unavailable."
            Config is written with placeholder. Doctor mode will flag it.

Read references/reference-flows.md for concrete auth flows from gh, gcloud, aws, railway, vercel, wrangler, supabase.

Rules for credential handling

1. Env var check comes first and blocks the wizard. If $TOOL_TOKEN is set, the interactive auth flow never runs. This is universal across gh, gcloud, aws, railway, vercel. The wizard should print what env var it found and exit.
2. Never store raw secrets in the config file. Store a reference: api_key_env = "ANTHROPIC_API_KEY" or api_key_cmd = "pass show anthropic" or api_key_keyring = "anthropic-api".
3. Always validate before accepting. A token found on disk may be expired or revoked. Ping the API.
4. On re-run, re-validate existing credentials. Doctor mode checks that stored creds are still valid. If expired, re-trigger the discovery cascade for that specific service only.
5. Multiple providers = multiple cascades. If the tool supports Claude + OpenAI + Mistral, run the cascade independently for each. The user might have Claude via OAuth but needs to paste a Mistral key.
6. OAuth must have a headless fallback. Three tiers (from gcloud): browser auto-open -> manual URL + code -> remote bootstrap command. Headless servers, SSH sessions, and containers can't open a browser.
7. Auto-skip when only one choice. If there is one provider, one account, one role -- select it automatically, don't ask. (AWS SSO pattern.)

Re-run credential flow

wizard --run (config exists, credentials present)
  |
  +-- doctor checks creds validity
  |     +-- Claude token: [PASS] valid, expires in 45 days
  |     +-- Mistral key:  [FAIL] 401 Unauthorized
  |     +-- OpenAI key:   [WARN] not configured (optional)
  |
  +-- "Mistral key is invalid. Reconfigure? [Y/n]"
  |     +-- Y -> run credential_discovery(mistral) cascade
  |
  +-- "Add OpenAI backend? [y/N]"
  |     +-- y -> run credential_discovery(openai) cascade
  |
  +-- "Modify other config? [backends / rules / all / skip]"

MCP credential tools

wizard_check_credentials()       -> { providers: [{name, status, expires_at, source}] }
wizard_discover_credential(svc)  -> runs cascade, returns { found: bool, source, valid }
wizard_set_credential(svc, ref)  -> stores credential reference, validates, returns { ok, valid }

Multi-Surface Architecture

One wizard engine, three surfaces. The engine writes the same config file regardless of entry point.

CLI (Huh/Gum/cliclack)  -+
MCP tool calls           -+---> wizard engine ---> config.toml ---> app reload
Web UI (Caddy-served)    -+

Surface: CLI

Surface: MCP

Each wizard step = one MCP tool. The agent calls tools sequentially, same engine.

wizard_get_config()          -> read current TOML, return struct
wizard_set_{section}(...)    -> modify one value, write TOML, return diff
wizard_run_doctor()          -> return pass/warn/fail per check
wizard_diff()                -> show pending changes before apply
wizard_apply()               -> reload app with new config

The MCP surface writes TOML, not JSON. JSON is only transport (tool call payloads, API responses). The user never sees JSON on disk.

Read references/mcp-patterns.md for MCP tool design patterns and MCP Apps UI spec.

Surface: Web UI

Pattern: Caddy Admin API model -- live config via REST, no restart.

• Optimistic concurrency via Etag / If-Match headers
• Validation before commit -- reject bad config, keep old running
• Atomic rollback on failure
• CLI wraps API: tool reload is sugar over curl -X POST /load

Server vs Client Mode

Server mode (remote + reverse proxy)

[wizard] -> generates:
  - app.toml           (backends, routing, rules)
  - Caddyfile          (TLS termination, reverse proxy -> app)
  - compose.yaml       (or k8s manifest)
  -> apply via SSH or Caddy Admin API

Client mode (local)

[wizard] -> generates:
  - ~/.app/config.toml  (server endpoint, API key, local port)
  -> app client points to remote server

Both modes use the same wizard engine. The mode is determined by one seed question or auto-detected.

Mitosis — Scale to scope

For tier S: skip the Multi-Surface Architecture section entirely.

Input

$ARGUMENTS is the target to audit:

• File (cmd/wizard.go, setup.py): audit that wizard implementation
• Directory (cmd/, src/): find and audit all wizard/setup flows
• Empty: scan the entire project for wizard patterns

Workflow

Step 0 -- Discover wizard surfaces

1. Find wizard/setup entry points:

   grep -r "wizard\|setup\|init\|configure\|doctor" --include="*.go" --include="*.rs" --include="*.py" --include="*.ts" -l
   

2. Find config file schemas/templates:

   glob: **/*.toml.example, **/*.yaml.example, **/config.*.toml, **/schema.json
   

3. Find MCP tool definitions:

   grep -r "wizard_\|config_\|setup_" --include="*.go" --include="*.rs" --include="*.py" --include="*.ts" -l
   

4. Find web config endpoints:

   grep -r "/config\|/api/config\|admin.*api" --include="*.go" --include="*.rs" --include="*.py" --include="*.ts" -l
   

Step 1 -- Audit the 4 Laws

For each wizard surface found, check compliance with each law:

Law 1 -- Ask once, derive the rest

• Count total questions asked
• For each question: can the answer be derived from a previous answer or environment detection?
• Flag derivable questions as violations
• Check: does the wizard detect existing environment (OS, installed tools, running services)?

Law 2 -- Defaults are decisions

• List all config keys generated
• For each: is there a default value? Is the default documented/justified?
• Flag keys with empty string, null, or placeholder defaults
• Check: can a user accept all defaults and get a working system?

Law 3 -- Recap before apply

• Does the wizard show a summary/diff before writing config?
• Is there a --dry-run flag?
• Can the user abort after recap?
• Is the diff human-readable (not raw JSON dump)?

Law 4 -- Config-as-code output

• Does the wizard generate a file on disk?
• Is the file in a committable format (TOML, YAML, not binary)?
• Can the wizard re-read an existing config and diff against it?
• Is the config file documented (comments, inline help)?

Step 2 -- Audit lifecycle completeness

Step 2b -- Audit credential discovery

Verify the wizard implements a credential discovery cascade (not just "paste your API key"):

Step 3 -- Audit doctor checks

Read references/doctor-checks.md for the standard check categories (including credentials).

For each check found in the doctor mode, verify:

• Clear pass/warn/fail status
• Actionable fix recommendation for each warn/fail
• Independent execution (one failing check doesn't block others)
• Exit code reflects overall status (0 = all pass, 1 = warnings, 2 = failures)
• Credential checks re-trigger discovery cascade on failure (not just "key invalid, fix it yourself")

Step 4 -- Audit multi-surface consistency

If the wizard has multiple surfaces (CLI + MCP, CLI + web):

• Do all surfaces write the same config file format?
• Do all surfaces use the same validation logic?
• Can a config written by one surface be read/edited by another?
• Is there a single source of truth (one config file, not scattered state)?

Step 5 -- Detect anti-patterns

Read references/anti-patterns.md for the named anti-patterns with detection heuristics.

Step 6 -- Audit scriptability

• Does every interactive prompt have a corresponding CLI flag?
• Can the wizard run fully non-interactive via flags/env vars?
• Is there a --yes / --accept-defaults flag for CI usage?
• Can config be piped in via stdin?

Step 7 -- Score and report

Finding tier and confidence semantics are canonical in ../shared/triage.md (Tier 3/2/1 + GRADE + triangulation). Law violations are Tier 3 by default; lifecycle gaps are Tier 2.

Output Format

# Wizard Audit -- {project-name}

**Date:** {date}
**Scope:** {what was analyzed}
**Wizard surfaces found:** {CLI / MCP / Web / count}
**Config format:** {TOML / YAML / JSON / other}
**Wizard Quality Index:** {X}/100

## Law Compliance

| Law | Status | Evidence |
|-----|--------|----------|
| 1. Ask once, derive the rest | {Pass/Warn/Fail} | {N} derivable questions found |
| 2. Defaults are decisions | {Pass/Warn/Fail} | {N} keys without meaningful defaults |
| 3. Recap before apply | {Pass/Warn/Fail} | Dry-run: {yes/no}, Recap: {yes/no} |
| 4. Config-as-code output | {Pass/Warn/Fail} | Format: {format}, Committable: {yes/no} |

## Lifecycle Coverage

| Mode | Status | Notes |
|------|--------|-------|
| Setup (config absent) | {Implemented / Missing} | |
| Doctor (config present) | {Implemented / Missing} | |
| Edit (section) | {Implemented / Missing} | |
| Migrate (schema version) | {Implemented / Missing} | |
| Dry-run | {Implemented / Missing} | |
| Idempotent re-run | {Pass / Fail} | |

## Doctor Checks ({N} found)

| Check | Category | Status | Fix suggestion |
|-------|----------|--------|---------------|
| Config schema valid | Schema | {pass/warn/fail} | |
| Backend reachable | Connectivity | {pass/warn/fail} | |
| TLS cert valid | Security | {pass/warn/fail} | |

## Multi-Surface Consistency

| Surface | Config format | Validation shared | Interoperable |
|---------|--------------|-------------------|---------------|
| CLI | {format} | {yes/no} | {yes/no} |
| MCP | {format} | {yes/no} | {yes/no} |
| Web | {format} | {yes/no} | {yes/no} |

## Anti-Patterns Detected

| # | Pattern | Severity | Evidence | Fix |
|---|---------|----------|----------|-----|
| 1 | Premature Interrogation | Major | Setup asks 12 questions, 8 derivable | Derive from first 4 |

## Scriptability

| Check | Status |
|-------|--------|
| All prompts have flags | {yes/no, N missing} |
| Non-interactive mode | {yes/no} |
| --yes / --accept-defaults | {yes/no} |
| Stdin config input | {yes/no} |
| CI-friendly exit codes | {yes/no} |

## Recommendations

| # | Action | Impact | Effort | Target |
|---|--------|--------|--------|--------|
| 1 | Add --dry-run flag | High | Low | cmd/wizard.go |
| 2 | Implement doctor mode | High | Medium | cmd/doctor.go |

Scoring

wqi = 100
  - (law_1_violations x 5)         # derivable questions
  - (law_2_violations x 5)         # empty/null defaults
  - (law_3_missing x 15)           # no recap/dry-run
  - (law_4_missing x 15)           # no config-as-code
  - (missing_lifecycle_modes x 8)  # setup/doctor/edit/migrate/dry-run
  - (anti_pattern_count x 7)       # named anti-patterns
  - (non_scriptable_prompts x 3)   # prompts without flags
  - (surface_inconsistency x 10)   # multi-surface divergence
  clamped to [0, 100]

Rules for the Auditor

1. UX over implementation. This skill audits the user experience of configuration, not the config file format itself. That's cli-audit-code's job.
2. Test the absent path. Always check what happens when config doesn't exist yet. First-run UX is the most critical moment.
3. Test the re-run path. A wizard that works once but breaks on re-run is worse than no wizard.
4. Scriptability is not optional. If a wizard can't run in CI, it's incomplete.
5. Doctor mode is not a nice-to-have. It's half the wizard. Setup without doctor is a one-shot anti-pattern.
6. Gotchas -- read ../gotchas.md before producing output to avoid known mistakes.

Integration with other cli-* skills

Dynamic Handoffs

Rule: Recommend, don't auto-execute.

What this skill does NOT do

• Does not design wizards -- it audits existing ones and recommends improvements
• Does not generate config files -- it checks that the wizard generates them correctly
• Does not replace user testing -- it catches structural issues, not subjective UX feel
• Does not audit config file syntax -- TOML/YAML validity is a linter's job