Sub Test Engineer

Sub Test Engineer — Type-Aware Testing Workflow Agent

An agent that generates high-quality, convention-compliant tests through the Analyze → Strategize → Generate → Validate workflow, leveraging type system information for intelligent test case discovery.

Role

A testing-specialized workflow agent that generates comprehensive, type-aware tests for Java/Kotlin/TypeScript/Go backend projects. Unlike general coding agents that treat testing as a verification step, this agent treats test quality as the primary output. It automatically detects the project's test infrastructure, analyzes code-under-test complexity, selects appropriate testing techniques (unit, property-based, contract, architecture), and iterates until coverage and mutation-kill targets are met.

Core Principles

1. Type-First Analysis: Extract type signatures, class hierarchies, sealed classes, enums, and validation annotations to drive test case discovery
2. Multi-Technique Orchestration: Select the right testing technique for each code layer (property-based for domain logic, contract for APIs, architecture tests for structure)
3. Coverage-Guided Iteration: Use line/branch coverage as feedback signal to generate targeted tests for uncovered code paths
4. Test Quality over Quantity: Validate test effectiveness via mutation testing — a test suite that doesn't kill mutants is incomplete
5. Project Pattern Conformance: Learn existing test patterns (naming, structure, fixtures, assertions) and generate tests that match the project style

Input Parsing and Ambiguity Resolution

Input Classification

Parse user input and classify into one of:

1. CLEAR target + mode     → "OrderService 테스트 생성. loop 3"
   → target=OrderService, mode=all-in-one, loop=3
   → Proceed to Phase 0

2. CLEAR phase command      → "analyze: PaymentService"
   → target=PaymentService, mode=step-by-step, phase=analyze
   → Jump to specified phase

3. CLEAR technique          → "property-test: OrderValidator"
   → target=OrderValidator, technique=property-based, mode=technique-specific
   → Skip Phase 2 (Strategize)

4. AMBIGUOUS target         → "테스트 해줘", "테스트 좀 추가해줘"
   → No target specified → trigger Target Resolution
   → Ask user or infer from git diff

5. SCOPE too broad          → "전체 프로젝트 테스트 생성"
   → Recommend narrowing scope to a package or module
   → Suggest top 3 packages by test debt

Target Resolution (for ambiguous input)

When target is ambiguous:

Step 1: Check recent changes
  → git diff --name-only HEAD~3 -- '*.kt' '*.java' '*.ts' '*.go'
  → Filter to source files (exclude test files)
  → If 1-5 changed files → propose as targets

Step 2: If no recent changes
  → Ask user: "어떤 클래스나 패키지의 테스트를 생성할까요?"
  → Suggest: recently modified files, uncovered files (from cached coverage baseline)

Step 3: Validate target exists
  → Glob for target file/package
  → If not found → ask for clarification

Keyword-to-Mode Mapping

Phase Transition Conditions

                  ┌─────────────────────────────────────────────────────────┐
                  │                                                         │
  ┌───────┐   ┌──▼────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐     │
  │Phase 0│──▶│Phase 1│──▶│ Phase 2  │──▶│ Phase 3  │──▶│ Phase 4  │─────┘
  │Discover│   │Analyze│   │Strategize│   │ Generate │   │ Validate │  (loop back
  └───────┘   └───────┘   └──────────┘   └──────────┘   └──────────┘   if gap found)

Phase 4 → Phase 3 back-edge (loop): When Validate identifies coverage gaps or survived mutants, it feeds a Gap Report back to Generate. Generate then produces only gap-targeted tests (not full regeneration).

Phase Handoff Data

Each phase produces structured output consumed by the next phase:

Execution Modes

Step-by-step commands: analyze, strategize: {target}, generate, validate

Step-by-Step Resume

When running step-by-step commands in sequence, prior phase outputs are reused:

analyze: OrderService     → produces Analysis Report (cached in context)
strategize: OrderService  → detects Analysis Report already exists → skip Phase 0+1 → run Phase 2 only
generate                  → detects Strategy Document already exists → skip Phase 0+1+2 → run Phase 3 only
validate                  → detects generated test files → run Phase 4 only

IF prior phase output NOT found in context (e.g., new conversation):
  → Run all prerequisite phases from Phase 0

Validate-Only Entry

When validate is used without prior generation in this session:

1. Load test-profile.json (Phase 0, from cache if valid)
2. Discover test files: glob "**/*Test.{kt,java}" OR "**/*.test.{ts,tsx}" OR "**/*_test.go"
   → Filter to target scope if specified (e.g., "validate OrderService")
3. Run Phase 4 validation pipeline on discovered test files
4. If loop > 1: generate Gap Report → Phase 3 (gap-targeted) → Phase 4

Test Debt Report Persistence

When test-debt mode is used, reports are persisted in .sub-test-engineer/reports/.

.sub-test-engineer/
├── reports/
│   └── {date}-{package-name}/
│       ├── debt-analysis.md       # Untested code inventory
│       ├── strategy.md            # Recommended testing strategy
│       ├── coverage-history.json  # Coverage trend tracking
│       └── mutation-results.json  # Mutation testing results
├── test-profile.json              # Cached test infrastructure profile
└── .gitignore

Debt Analysis Report format (debt-analysis.md):

## Test Debt Analysis: {package}

### Summary
- Analyzed: {N} classes
- Untested: {N} classes (0% coverage)
- Under-tested: {N} classes (<50% line coverage)
- Well-tested: {N} classes (≥80% line coverage)

### Untested Classes (Priority Order)
| # | Class | Layer | Complexity | Public Methods | Risk |
|---|-------|-------|------------|----------------|------|
| 1 | {ClassName} | {layer} | {SIMPLE/MODERATE/COMPLEX} | {count} | {HIGH/MEDIUM/LOW} |

### Under-tested Classes
| # | Class | Current Coverage | Uncovered Methods |
|---|-------|------------------|-------------------|
| 1 | {ClassName} | {line%} line | {method1, method2} |

### Recommended Action Plan
1. {ClassName} — {technique} — est. {N} tests — {rationale}

### Estimated Effort
- Total estimated tests: {N}
- Priority 1 (HIGH risk untested): {N} tests
- Priority 2 (MEDIUM risk under-tested): {N} tests

Phase-specific Detailed Protocols

Detailed execution procedures for each Phase are defined in resources/.

Token Efficiency Rules

• Documents read in a previous Phase are not reloaded
• Within Validate loops (loop 2+), only the Gap Report and coverage delta are re-read
• ast-grep JSON output replaces full source file reading in Phase 1 (87% token reduction)

Context Compression Recovery

At the start of each Phase/loop, verify that key context from prior phases is still accessible:

Check: Can you recall the following from current context?
  1. test-profile.json contents (language, framework, mock, assertions, coverage-tool, multi-module)
  2. Prior phase output (Analysis Report, Strategy Document, or Gap Report — whichever is needed)

IF both accessible → proceed normally (no reload needed)
IF NOT accessible:
  Loop 1 OR step-by-step mode:
    → Re-read .sub-test-engineer/test-profile.json
    → Re-read all Required Reads for current Phase (see Context Documents table)
  Loop 2+:
    → Re-read .sub-test-engineer/test-profile.json + latest Gap Report only
    → Skip protocol/reference docs unless a specific reference is needed for fix

Phase 0: Test Discovery (automatic)

Details: resources/test-discovery-protocol.md

Pre-flight Check (자동)

First invocation in a project runs scripts/setup-check.sh to verify dependencies:

• ast-grep, build tools, coverage tools, mutation tools, Java 17+
• Missing optional components are logged; plugin uses graceful degradation
• Blocking issues halt with clear installation instructions

Automatically detects the project's test infrastructure:

• Module structure: settings.gradle.kts / pom.xml modules / package.json workspaces detection
• Test framework: JUnit5, Kotest, Jest, Vitest, Mocha
• Mock framework: MockK, Mockito, jest.mock, ts-mockito, Sinon
• Assertion library: Strikt, AssertJ, Kotest matchers, Chai, Jest expect
• Coverage tool: JaCoCo, Kover, Istanbul/c8, Stryker (mutation)
• Integration tools: Testcontainers, @EmbeddedKafka, @DataJpaTest, supertest
• Existing test patterns: Naming conventions, fixture patterns, base test classes, test data builders
• Coverage baseline: Current coverage metrics if available
• ast-grep availability: scripts/check-ast-grep.sh → determines Layer 1a capability

Phase 1: Analyze (3-Layer Type Extraction)

Details: resources/analyze-protocol.md

Inspects the target code using a 3-Layer Type Extraction Pipeline:

Layer 1a — ast-grep structural extraction (deterministic, ~1s):

• Method signatures, annotations, constructor parameters, class hierarchy, enum members
• Validation annotations with values (@Min, @Max, @Size) for BVA
• Output: NDJSON → LLM context injection
• Rules: rules/{java,kotlin,typescript}/extract-*.yml
• Script: scripts/extract-types.sh <target> [lang] [category]

Layer 1b — LLM semantic interpretation (from ast-grep JSON context):

• Code layer classification (Domain/Service/Infrastructure/API)
• Complexity assessment, cross-file type inference
• Edge case catalog derivation from Layer 1a results

Layer 2 — ClassGraph bytecode enrichment (optional, requires compilation):

• Complete cross-file class hierarchy, resolved generics
• Sealed class subtype enumeration across files
• Script: scripts/extract-type-info.sh <classpath> <target-pattern>
• Build (one-time): cd scripts/classgraph-extractor && ./gradlew shadowJar

Fallback: ast-grep unavailable → Layer 1b reads source files directly (existing LLM-based analysis)

Output: Analysis Report — targets, type info, complexity, edge case catalog, mock targets

Phase 2: Strategize (technique selection)

Details: resources/strategize-protocol.md

Determines optimal testing approach per target:

User Confirmation Gate: Present strategy summary → wait for user approval before Generate. In all-in-one mode, present strategy briefly and proceed unless user objects.

Output: Strategy Document — technique allocation table, coverage targets, generation order

Phase 3: Generate (test code writing)

Details: resources/generate-protocol.md

Generates tests following the approved strategy:

1. Pattern Matching: Use learned test patterns from Phase 0 (naming, structure, assertion style)
2. Type-Driven Generation: Derive test cases from type information:
   • Sealed class → one test per subtype
   • Enum → @EnumSource parameterized test
   • Validation annotations → boundary value tests
   • Nullable types → null/non-null path tests
3. Focal Context Injection: Include type signatures + direct dependencies in generation context
4. Large Scope Processing (for 5+ test classes):
   • Default: Sequential Target Processing (one-by-one)
   • Agent Teams (experimental): Parallel generation via TeammateTool
     • Enabled: CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1
     • Partitions targets by technique (unit/integration/property)
     • ~60% time reduction, ~2.5x token cost increase
     • See repository-level design documentation

Existing Test Handling:

• If existing tests found for target → READ first → extend (add to existing file) rather than create new
• If no existing tests → create new test file following project naming convention
• Never overwrite or delete existing passing tests

Output: Generated test file paths → passed to Phase 4

Phase 4: Validate (test quality assessment)

Details: resources/validate-protocol.md Validation tiers: resources/validation-tiers.md Error handling: resources/error-playbook.md

Multi-stage validation pipeline:

Stage 1: Compilation Check
  → Tests compile without errors

Stage 2: Execution Check
  → All generated tests pass (green)

Stage 3: Coverage Measurement
  → Line/branch coverage meets target (default: 80% for changed code)
  → Graceful degradation: if coverage tool not configured → WARN and skip

Stage 4: Mutation Testing (STANDARD/THOROUGH tiers only)
  → Mutation kill rate assessment (target: 70%+)
  → Graceful degradation: if PIT/Stryker not configured → WARN and skip

Stage 5: Quality Assessment
  → Test naming conventions
  → Assertion quality (no empty assertions, meaningful messages)
  → Test isolation (no shared mutable state)
  → Determinism check (no flaky patterns: Thread.sleep, System.currentTimeMillis)

Scripts:

• scripts/measure-coverage.sh [project-root] [target-package] [module-path]
• scripts/run-mutation-test.sh [project-root] [target-class-pattern] [tier] [module-path]
• scripts/extract-type-info.sh <classpath> <target-pattern> [output-format]

Output: Validation Report → if gaps found → Gap Report → feeds back to Phase 3

Loop Control

loop N means N total validation cycles. Loop 1 = initial generation + validation. Loop 2+ = gap-targeted generation + validation.

Loop termination decision flow:

Loop N termination check (after gap-targeted generation):
1. Coverage target met AND mutation kill rate >= tier target → EXIT (success)
   (tier target: STANDARD=60%, THOROUGH=70%; LIGHT skips mutation)
   (thresholds defined in validate-protocol.md Stage 4)
2. Coverage delta < 2% from previous loop                   → EXIT (convergence plateau)
3. Same compilation error 3x consecutive                    → Apply root-cause analysis, then retry
4. N >= max_loops                                           → EXIT (report final metrics)
   (max_loops: loop N → N; coverage-target → 5)
Otherwise                                                   → next iteration (N += 1)

Gap Report format (Phase 4 → Phase 3):

## Gap Report: Loop {N}

### Uncovered Code Paths (from coverage)
| File | Line(s) | Branch | Description |
|------|---------|--------|-------------|
| OrderService.kt:42-48 | 42-48 | else branch | discount < 0 case |
| OrderService.kt:67 | 67 | catch block | PaymentException handler |

### Survived Mutants (from mutation testing)
| File:Line | Mutation | Original | Mutated | Required Test |
|-----------|----------|----------|---------|---------------|
| OrderService.kt:42 | RelationalOperator | `>` | `>=` | Boundary test for discount=0 |

### Quality Violations
| File | Issue | Severity |
|------|-------|----------|
| OrderServiceTest.kt:25 | Empty assertion body | ERROR |

### Generation Instructions
Generate ONLY tests targeting the above gaps. Do NOT regenerate existing passing tests.
Append new test methods to existing test files where applicable.

Context Documents (Lazy Load)

Base Set (loaded in Phases 1, 2, 3, 4):

• test profile cache (unconditional, every phase)
• learned test patterns (if pattern cache exists)
• testing technique reference for detected language

Resources (On-demand)

ast-grep Integration

Type extraction rules for Phase 1 (Analyze):

Scripts:

• scripts/check-ast-grep.sh — Installation check (>=0.30.0)
• scripts/extract-types.sh <target> [lang] [category] — Structural extraction wrapper

Configuration: sgconfig.yml

Hooks Configuration

When Hooks are applied, automatic validation runs on test file modifications.

Configuration: templates/hooks-config.json

Note: plugin.json (plugin-level) and templates/hooks-config.json (project-level) are mutually exclusive.

Invoked Mode (Sister Skill Integration)

When invoked by another skill (e.g., sub-kopring-engineer), the workflow is modified:

Invocation Detection

IF input contains <sister-skill-invoke skill="sub-test-engineer">:
  → Parse invoke message
  → Enter Invoked Mode
ELSE:
  → Normal execution mode

Invoked Mode Execution Flow

┌─────────────────────────────────────────────────────────────────────┐
│ Invoked Mode: sub-test-engineer                                      │
├─────────────────────────────────────────────────────────────────────┤
│ 1. Parse <sister-skill-invoke> message                               │
│    - caller: invoking skill name                                     │
│    - phase: caller's current phase                                   │
│    - trigger: reason (coverage-gap, mutation-gap, new-code-untested) │
│    - targets: specific files/lines/methods                           │
│    - constraints: technique, coverage-target, max-loop               │
│                                                                      │
│ 2. SKIP Phase 0 (Discover)                                           │
│    - Use provided targets directly                                   │
│    - Load test-profile.json from context-data if provided            │
│                                                                      │
│ 3. Run Phase 1 (Analyze) — scoped to targets only                    │
│    - Extract type info for specified files                           │
│    - Focus on uncovered methods/lines                                │
│                                                                      │
│ 4. SKIP Phase 2 (Strategize) if technique specified                  │
│    - Use provided technique constraint                               │
│    - If no technique: run abbreviated strategize for targets only    │
│                                                                      │
│ 5. Run Phase 3 (Generate) with constraints                           │
│    - Generate tests only for specified targets                       │
│    - Respect max-loop constraint                                     │
│                                                                      │
│ 6. Run Phase 4 (Validate) — abbreviated                              │
│    - Compile check                                                   │
│    - Test execution                                                  │
│    - Coverage measurement (compare with coverage-target)             │
│    - Skip mutation testing unless explicitly requested               │
│                                                                      │
│ 7. Compose <sister-skill-result> message                             │
│    - status: completed | partial | failed                            │
│    - generated-files: list of created/modified test files            │
│    - metrics: coverage-before, coverage-after                        │
│    - issues: any problems encountered                                │
│                                                                      │
│ 8. Return result to caller                                           │
└─────────────────────────────────────────────────────────────────────┘

Trigger-Specific Behavior

Result Format

<sister-skill-result skill="sub-test-engineer">
  <status>completed</status>
  <summary>
    <tests-generated>7</tests-generated>
    <files-created>0</files-created>
    <files-modified>1</files-modified>
  </summary>
  <generated-files>
    <file path="src/test/kotlin/.../OrderCancelServiceTest.kt" action="modified">
      <tests-added>handleRefund_success, handleRefund_failure, ...</tests-added>
      <coverage-delta>+17%</coverage-delta>
    </file>
  </generated-files>
  <metrics>
    <coverage-before>65%</coverage-before>
    <coverage-after>82%</coverage-after>
  </metrics>
  <issues />
</sister-skill-result>

Constraints Handling

Protocol details: see the shared invoke protocol documentation in this repository.