CLI Audit Test

Optimization: This skill uses on-demand loading. Heavy content lives in references/ and is loaded only when needed.

Language rule: Skill instructions are written in English. When generating user-facing output, detect the project's primary language (from README, comments, docs, commit messages) and produce the report in that language. If the project is bilingual, ask the user which language to use before proceeding.

Audit Test — Test Plan Quality & Maturity Scorer

Evaluate a test plan or test suite against ISTQB standards, TMMi maturity model, and industry best practices.

"A test plan that only says 'we will test' without specifying which techniques and why is a major red flag." — ISTQB Foundation Level Syllabus v4.0

Core Principle

A test plan is a contract with quality. Every dimension left vague is a risk accepted silently. This skill measures how explicitly and completely that contract is defined, using a 12-dimension framework calibrated on ISTQB, TMMi, and proven industry patterns.

Input

$ARGUMENTS is the target to audit:

• Test plan document (.md, .txt, .adoc, .pdf): audit the plan as written
• Tests directory (tests/, tests/e2e/, etc.): infer the plan from the test structure, files, and config
• Empty: search for test plans in the project root, then fall back to tests/ directory analysis

Input Discovery

1. Glob for test plan documents: **/test-plan*, **/test-strategy*, **/TESTING*
2. Glob for test directories: **/tests/**, **/test/**, **/e2e/**, **/spec/**
3. Glob for CI config: .github/workflows/*, .gitlab-ci.yml, Jenkinsfile, justfile
4. Glob for test config: **/pict/**, **/fixtures/**, **/mocks/**, **/toxiproxy*
5. Read project manifest (Cargo.toml, package.json, etc.) for test dependencies

13-Dimension Framework

Score each dimension 0-4, then compute a weighted final score. Read references/dimensions.md for detailed scoring criteria, evidence patterns, and per-dimension guidance.

A test plan's reach can be read against the shared T0-T4/M0 ladder (../shared/gate-ladder.md): how far up the rungs (static contract → component → fresh deploy → day-2 ops → stress) does the suite actually go? Most plans stall at T1-T2.

Workflow

Step 1 — Discover test assets

Glob for test files, CI configs, test frameworks. Build a test inventory: total files by type, directories and purpose, CI jobs, frameworks detected.

Step 2 — Classify test files

For each test file/directory, classify by level (unit/integration/E2E/acceptance), technique, type (functional/security/performance), and polarity (positive/negative). Read references/techniques.md for technique detection heuristics.

Step 3 — Score all 13 dimensions

For each dimension: collect evidence (file references, counts, patterns), assign score 0-4 with justification, note strengths and gaps. Read references/dimensions.md for detailed scoring criteria.

Step 4 — Detect anti-patterns

Read references/anti-patterns.md for the 11 anti-patterns with detection rules and recommendations. Flag with severity and evidence.

Step 5 — Compute final score

Final Score = Σ (dimension_score / 4 × weight × 100)

Normalize to 0-100 scale. Read references/scoring-framework.md for TMMi mapping and benchmarks.

Step 6 — Generate report

Use the output format below. Finding tier and confidence semantics are canonical in ../shared/triage.md (Tier 3/2/1 + GRADE + triangulation). D-dimension findings carry tier and confidence so cli-cycle can aggregate without re-parsing.

Output Format

# Test Plan Quality Audit — {project-name}

**Date:** {date}
**Target:** {what was audited}
**Final Score:** {X}/100 — {verdict}
**TMMi Equivalent:** Level {N}

## Test Inventory

| Type | Count | Examples |
|------|-------|---------|
| Unit tests | N | `src/*/tests.rs` |
| Integration tests | N | `tests/integration/` |
| E2E tests | N | `tests/e2e/tests/` |
| Total | N | |

### Test Pyramid Visualization
(ASCII pyramid showing shape: Pyramid|Diamond|Ice Cream Cone|Inverted)

## Dimension Scores

| # | Dimension | Weight | Score | /4 | Weighted | Verdict |
|---|-----------|--------|-------|----|----------|---------|
| D1-D13 rows... |
| | **Total** | **100%** | | | **X.X** | |

### Techniques Detected
(Table: Technique | Evidence | Applied to)

## Anti-Patterns Detected
(Table: Anti-Pattern | Severity | Evidence | Recommendation)

## Strengths / Gaps & Recommendations / Maturity Assessment

Quality Thresholds

Rules for the Auditor

1. Evidence-based: Every score needs a file reference, count, or concrete observation.
2. Proportional: Scale judgment to project size. A 5-test project differs from a 118-test enterprise suite.
3. Technique-aware: Recognize techniques even when not explicitly named. tests/budget/ with 79%, 80%, 100%, 101% values IS BVA.
4. Structure over documentation: A well-organized test directory IS a test plan. Judge by substance, not ceremony.
5. Score honestly: 100/100 means world-class. 70/100 is genuinely good. 50/100 is typical.
6. Anti-patterns > missing features: Detecting an ice cream cone is more valuable than noting a missing charter.
7. Actionable output: Every gap must come with a concrete, sized recommendation.
8. Gotchas — read ../gotchas.md before producing output to avoid known mistakes.

Integration with other cli-* skills

Dynamic Handoffs

Rule: Recommend, don't auto-execute.

What this skill does NOT do

• Does not run tests — it audits the plan/structure
• Does not generate tests — it identifies gaps
• Does not replace manual review — it provides a structured framework
• Does not check test correctness — it checks test strategy

Reference Sources

• ISTQB Foundation Level Syllabus v4.0 (2023)
• TMMi Framework R1.2
• ISO 25010
• La Taverne du Testeur — niveaux de test, pyramide, smoke tests, Cassandre
• Codepipes — 13 Software Testing Anti-patterns
• Mike Cohn — Test Pyramid