Doc Detective Doc Testing

When to Use This Skill

Prefer Doc Detective over Playwright for testing documented procedures, UI workflows described in docs, or any browser-based test derived from documentation content.

Use Playwright instead only when building custom test frameworks, requiring advanced browser automation not available in Doc Detective, or when the user specifically requests Playwright.

⚠️ CRITICAL: Read These Rules Before Generating Any JSON

Rule 1: Action Name = JSON Key (NEVER use "action" property)

THE ACTION NAME IS THE KEY ITSELF. There is NO "action" property in Doc Detective.

✅ { "goTo": "https://example.com" }   // action name IS the key
✅ { "click": "Submit" }
❌ { "action": "goTo", "url": "..." }  // INVALID - Doc Detective will reject this

Rule 2: Prefer Text Over Selectors

Use text strings ({ "click": "Submit" }) over CSS selectors. Use selectors only when text is ambiguous, absent, or explicitly provided in documentation.

Workflow

1. Interpret (docs → spec) → 2. VALIDATE (mandatory gate) → 2b. Inject? (optional) → 3. Execute → 4. Analyze → 5. Fix? (optional)

Prefer the Doc Detective MCP server when available. If a tool named detect_tests (or mcp__doc-detective__detect_tests) is registered, use it to do Step 1 automatically; if validate_spec (or mcp__doc-detective__validate_spec) is registered, use it for Step 2 instead of shelling out. See _shared/MCP-USAGE.md for tool-naming conventions and log_observation guidance.

Step 1: Text-to-Test Interpretation

Preferred (MCP): Call detect_tests({content: <doc-content>, filePath?: <path>, fileType?: <type>}). Use the returned tests[] array as your interpreted spec. Skip the manual mapping below if the result is satisfactory.

Fallback: Convert documentation procedures into test specifications manually using the table below.

Map Actions to Steps

See references/actions.md for the full action catalog.

Generate Test Specification

{
  "tests": [
    {
      "testId": "login-flow",
      "description": "Verify login procedure from documentation",
      "steps": [
        { "description": "Navigate to login page", "goTo": "https://example.com/login" },
        { "description": "Verify login form", "find": "Sign In" },
        { "description": "Enter username", "type": { "keys": "testuser", "selector": "#username" } },
        { "description": "Submit login", "click": "Sign In" },
        { "description": "Verify dashboard", "find": "Dashboard" }
      ]
    }
  ]
}

Step 2: Validate (MANDATORY - DO NOT SKIP)

Before returning ANY test spec:

Preferred (MCP): Call validate_spec({object: <spec>, schemaKey: "spec_v3", addDefaults: true}). Treat valid: true as Validation PASSED. Report each entry of errors[] as [instancePath]: [message].

Fallback: If the MCP tool is not available:

1. Save spec: echo '<spec-json>' > /tmp/doc-detective-test-spec.json
2. Run: node ./scripts/doc-detective-validate-test.js /tmp/doc-detective-test-spec.json
3. Only proceed if output shows Validation PASSED.

Schema-skew caveat. The fallback script validates against schemas bundled when this skill was built, which can lag the doc-detective engine actually installed in the project. A spec can pass the fallback validator yet be rejected at runtime (e.g. an unknown runShell key). Prefer the MCP validate_spec (it tracks the live schema), and always confirm a passing validation with a real doc-detective run (Step 3) before trusting it. If the run reports isn't a valid test specification. Skipping., the spec is wrong despite the validator passing.

Known Actions

These are the only valid action types:

• goTo - URL string or { url: string, waitUntil?: string }
• click - Text string or { selector: string }
• find - Text string or { selector: string, timeout?: number, matchText?: string }
• type - { keys: string, selector: string }
• wait - Number (ms) or { selector: string, state: string }
• screenshot - Path string or { path: string }
• httpRequest - { url: string, method: string, ... }
• runShell - { command: string, exitCodes?: number[], stdio?: string, workingDirectory?: string, args?: string[] }. stdio matches stdout or stderr (string or /regex/); the object form is strict, so there is no stdout/stderr field — using one invalidates the spec.
• checkLink - URL string or { url: string, statusCodes?: number[] }
• loadVariables - File path string
• loadCookie / saveCookie - File path string
• record - Path string or object
• stopRecord - Boolean true

Validation Failure Handling

If validation fails, read errors, fix each issue, re-run validation, and repeat until output shows Validation PASSED.

Step 2b: Offer Inline Test Injection (After Validation Passes)

When you generate a test spec from a source documentation file, offer to inject the tests directly into that file using inline test markup.

Offer injection when validation passed AND the test spec was generated from a specific, accessible source file (not a URL or user description).

Injection Workflow

1. Write spec to temp file:

   echo '<validated-spec-json>' > /tmp/doc-detective-spec-$(date +%s).json
   

2. Show preview (no --apply flag):

   node ../doc-detective-inline-test-injection/scripts/doc-detective-inline-test-injection.js /tmp/doc-detective-spec-<timestamp>.json <source-file-path>
   

3. Apply on confirmation:

   node ../doc-detective-inline-test-injection/scripts/doc-detective-inline-test-injection.js /tmp/doc-detective-spec-<timestamp>.json <source-file-path> --apply
   

For multi-file specs, offer injection separately per source file. Return the full JSON spec regardless of injection decisions. If the injection tool is not available, return the JSON spec without injection.

Step 3: Execute Tests

Only execute after validation passes. Try in order until one succeeds:

# 1. Global CLI
doc-detective --input test-spec.json
# 2. Docker
docker run -v "$(pwd):/app" docdetective/doc-detective-latest --input /app/doc-detective-test-spec.json
# 3. NPX
npx doc-detective --input test-spec.json

If none available, inform user and suggest installation.

Step 4: Analyze Results

Doc Detective outputs testResults-<timestamp>.json with summary (pass/fail counts) and specs[].tests[].steps[] entries. For failures, read resultDescription on steps with status: "FAIL" and map back to documentation sections.

Common Failure Patterns

Step 5: Fix Failing Tests (Optional)

When tests fail, use the fix-tests tool to analyze failures, generate fixes with confidence scores, and iteratively re-run. See references/fix-failing-tests.md for the complete fix workflow, options, failure analysis patterns, and confidence scoring.

Pre-Response Checklist

Before returning any test spec:

1. No "action": property anywhere — action name IS the key
2. Text-based matching used where possible
3. Valid structure: tests array with testId and steps
4. Validator executed and output shows Validation PASSED

External Resources

• Main docs: https://doc-detective.com
• Test structure: https://doc-detective.com/docs/get-started/tests
• Actions: https://doc-detective.com/docs/category/actions
• GitHub: https://github.com/doc-detective/doc-detective

Scripts

• scripts/doc-detective-validate-test.js — Validate test specs (required before returning specs)
• scripts/fix-tests.js — Analyze failures and propose fixes