Bug Hunt
You are conducting an exhaustive bug hunt on this Git repository, enhanced by Draft context when available.
Primary Deliverable
The bug report is the primary deliverable. Every verified bug MUST appear in the final report regardless of whether a regression test can be written. Regression tests are a supplementary output — helpful when possible, but never a filter for bug inclusion.
Relationship to Built-in Bug Hunt Agents
Some AI tools (e.g., Claude Code) provide a built-in bughunt agent that auto-discovers project structure and runs parallel sweeps. /draft:bughunt is complementary, not competing:
/draft:bughunt |
Built-in bughunt agent | |
|---|---|---|
| Approach | Context-driven methodology with 14 analysis dimensions and verification protocol | Auto-discovery with parallel sweep subagents |
| Draft context | Uses architecture, tech-stack, product, guardrails for false-positive elimination | No Draft context awareness |
| Output | Severity-ranked report with evidence | Inline fixes + regression tests |
| Modifies code | No (report + regression tests only) | Yes (finds AND fixes) |
When to use which: Use /draft:bughunt when you need context-aware analysis with structured evidence and false-positive elimination. Use the built-in agent when you want fast parallel sweeps with auto-fix capability. For maximum coverage, run both — /draft:bughunt catches context-specific bugs the built-in misses, and vice versa.
Red Flags - STOP if you're:
- Hunting for bugs without reading Draft context first (architecture.md, tech-stack.md, product.md)
- Reporting a finding without reproducing or tracing the code path
- Fixing production code instead of reporting bugs (bughunt reports bugs and writes regression tests — it doesn't fix source code)
- Assuming a pattern is buggy without checking if it's used successfully elsewhere
- Skipping the verification protocol (every bug needs evidence)
- Making up file locations or line numbers without reading the actual code
- Reporting framework-handled concerns as bugs without checking the docs
- Skipping bugs because you can't write a test for them — mark as N/A and still report
Verify before you report. Evidence over assumptions.
Pre-Check
0. Capture Git Context
Before starting analysis, capture the current git state:
git branch --show-current # Current branch name
git rev-parse --short HEAD # Current commit hash
Store this for the report header. All bugs found are relative to this specific branch/commit.
1. Load Draft Context (if available)
Read and follow the base procedure in core/shared/draft-context-loading.md.
Bug-hunt-specific context application:
- Flag violations of intended architecture as bugs (coupling, boundary violations)
- Apply framework-specific checks from tech-stack (React anti-patterns, Node gotchas, etc.)
- Catch bugs that violate product requirements or user flows
- Prioritize areas relevant to active tracks
- Leverage Critical Invariants — Check for invariant violations across data safety, security, concurrency, ordering, idempotency categories
- Leverage Concurrency Model — Use thread/async model info for race condition and deadlock analysis
- Leverage Error Handling — Use failure modes and retry policies for reliability bug detection
- Leverage Data State Machines — Check for invalid state transitions, missing guard clauses, states with no exit path
- Leverage Storage Topology — Identify data loss risks at each tier (cache eviction without writeback, event log gaps, missing archive)
- Leverage Consistency Boundaries — Find bugs at eventual consistency seams (stale reads, lost events, missing reconciliation)
- Leverage Failure Recovery Matrix — Verify idempotency claims, check for partial failure states without recovery paths
2. Confirm Scope
When invoked programmatically by /draft:review with with-bughunt, skip scope confirmation and inherit the scope from the calling command.
Otherwise, ask user to confirm scope:
- Entire repo - Full codebase analysis
- Specific paths - Target directories or files
- Track-level (specify
<track-id>) - Focus on files relevant to a specific track
3. Load Track Context (if track-level)
If running for a specific track, also load:
-
draft/tracks/<id>/spec.md- Requirements, acceptance criteria, edge cases -
draft/tracks/<id>/plan.md- Implementation tasks, phases, dependencies
Use track context to:
- Verify implemented features match spec requirements
- Check edge cases listed in spec are handled
- Identify bugs in areas touched by the track's plan
- Focus analysis on files modified/created by the track
If no Draft context exists, proceed with code-only analysis.
Dimension Applicability Check
Before analyzing all 14 dimensions, determine which apply to this codebase:
- Skip explicitly rather than forcing analysis of N/A dimensions
- Mark skipped dimensions with reason in report summary
Examples of skipping:
- "N/A - no backend code" (skip dimensions 2, 8, 10 for frontend-only repo)
- "N/A - no UI components" (skip dimensions 5, 9, 14 for CLI tool)
- "N/A - no database" (skip dimension 2 for in-memory app)
- "N/A - no external integrations" (skip dimension 8)
- "N/A - no external dependencies" (skip dimension 12 for zero-dependency project)
- "N/A - no user-facing strings" (skip dimension 14 for libraries/APIs)
Analysis Dimensions
Analyze systematically across all applicable dimensions. Skip N/A dimensions explicitly (see Dimension Applicability Check above).
1. Correctness
- Logical errors, invalid assumptions, edge cases
- Incorrect state transitions, stale or inconsistent UI state
- Error handling gaps, silent failures
- Off-by-one errors, boundary conditions
2. Reliability & Resilience
- Crash paths, unhandled exceptions
- Reload/refresh behavior, retry logic
- UI behavior on partial backend failure
- Broken recovery after errors, navigation
3. Security
- XSS, injection vectors, unsafe rendering
- Client-side trust assumptions
- Secrets, tokens, auth data exposure
- CSRF, insecure deserialization
- Path traversal, command injection
- Taint tracking (end-to-end data flow analysis):
- Identify all entry points: HTTP params, form data, file uploads, env vars, CLI args, message queue payloads, webhook bodies
- Trace user input to dangerous sinks: SQL queries, shell exec, eval, innerHTML, file path construction, URL construction, deserialization, template rendering
- For each sink, verify sanitization/validation exists on every path from source to sink
- Flag paths where unsanitized input reaches a sink without passing through a validator, encoder, or sanitizer
- Reference: OWASP Top 10, Meta Infer taint analysis methodology
4. Performance (Backend + UI)
- Inefficient algorithms and data fetching
- Blocking work on main/UI thread
- Excessive re-renders, unnecessary state updates
- Unbounded memory growth (listeners, caches, stores)
5. UI Responsiveness & Perceived Performance
- Long tasks blocking input
- Jank during scrolling, typing, resizing
- Layout thrashing, forced reflows
- Expensive animations or transitions
- Poor loading states, flicker, content shifts
6. Concurrency & Ordering
- Race conditions between async calls
- Stale responses overwriting newer state
- Incorrect cancellation or debouncing
- Event ordering assumptions
- Deadlocks, livelocks
7. State Management
- Source-of-truth violations
- Derived state bugs (computed from stale data)
- Global state misuse
- Memory leaks from subscriptions or observers
- Inconsistent state across components
8. API & Contracts
- UI assumptions not guaranteed by backend
- Schema drift, weak typing, missing validation
- Backward compatibility risks
- Undocumented API behavior dependencies
9. Accessibility & UX Correctness
- Keyboard navigation gaps
- Focus management bugs
- ARIA misuse or absence
- Broken tab order or unreadable states
- UI behavior that contradicts user intent
- Color contrast, screen reader compatibility
10. Configuration & Build
- Fragile environment assumptions
- Build-time vs runtime config leaks
- Dev-only code shipping to prod
- Missing environment variable validation
- CI gaps affecting builds or tests
11. Tests
- Missing coverage for critical flows
- Snapshot misuse (testing implementation, not behavior)
- Tests that assert implementation instead of behavior
- Mismatch between test and real user interaction
- Flaky tests, timing dependencies
- Property-based testing gaps: pure/mathematical functions without invariant-based tests (e.g.,
encode(decode(x)) == x, sorting idempotency, associativity) - Test isolation violations: shared mutable state between test cases (global variables, singletons, class-level state modified in tests without reset)
- Test double misuse: mocks that leak state across tests, over-mocking (>3 mocks per test suggests testing wiring not behavior), stubs that diverge from real implementation behavior
- Assertion density: tests with zero or weak assertions (
assertTrue(true),expect(result).toBeDefined()only, empty catch blocks in test code,assert result is not Noneas sole check) - Flaky test patterns: time-dependent assertions (sleep, Date.now, timestamps), port/file system assumptions, test ordering dependencies, non-deterministic data (random seeds, UUIDs without control)
12. Dependency & Supply Chain Security
- Known CVEs: Check dependencies against known vulnerability databases (reference tools: Snyk, Trivy, OWASP Dependency-Check,
npm audit,pip-audit,cargo audit,go vuln) - Unpinned dependency versions: Lockfile freshness, use of version ranges (
^,~,*,>=) without lockfile enforcement, missing lockfile entirely - Deprecated packages: Dependencies with known deprecation notices, archived repositories, or no maintenance activity
- License conflicts: GPL dependencies in MIT/Apache projects, AGPL in proprietary code, incompatible license combinations in the dependency tree
- Typosquatting risk: Packages with names similar to popular ones (e.g.,
lodahsvslodash,reqeustsvsrequests), recently published packages with few downloads - Transitive dependency depth: Deeply nested dependency chains (>5 levels) increase supply chain attack surface; flag packages that pull in disproportionate transitive trees
- Reference: Google OSS-Fuzz, Microsoft SDL, OpenSSF Scorecard
13. Algorithmic Complexity
- Quadratic or worse loops: O(n^2) or worse nested loops over collections (nested
.filter()inside.map(), repeated linear scans, cartesian joins in application code) - Regex catastrophic backtracking: Nested quantifiers (
(a+)+,(a|a)*), unbounded repetition with overlapping alternatives — flag any regex applied to user-controlled input - Unbounded recursion: Recursive functions without depth limits, missing base cases, or base cases that depend on external/mutable state
- Cache invalidation storms: Cache miss triggering expensive recomputation that itself invalidates caches, thundering herd on cache expiry without jitter/locking
- Hot path inefficiency: Sorting/searching in hot paths without appropriate data structures (linear scan where hash map suffices, repeated sorting of same collection, string concatenation in loops)
14. Internationalization & Localization
- Hardcoded user-facing strings: Strings displayed to users embedded directly in source code rather than externalized to resource files/i18n frameworks
- Locale-sensitive operations without locale parameter: String comparison (
<,>,localeComparewithout locale), date formatting (toLocaleDateStringwithout explicit locale), number formatting, sorting (alphabetical sort that assumes ASCII ordering) - RTL layout issues: Hardcoded LTR assumptions in UI code (absolute
left/rightpositioning, directional margin/padding, text alignment assumptions) - Unicode handling bugs: String length vs byte length confusion, missing normalization (NFC/NFD), emoji handling (multi-codepoint sequences split incorrectly,
string.lengthvs grapheme count), surrogate pair handling in substring operations
Bug Verification Protocol
CRITICAL: No bug is valid without verification. Before declaring any finding as a bug, complete ALL applicable verification steps:
Verification Checklist (for each potential bug)
Code Path Verification
- Read the actual code at the suspected location
- Trace the data flow from input to the bug location
- Check if there are guards, validators, or error handlers upstream
- Verify the code path is actually reachable in production
Context Cross-Reference
- Check
.ai-context.md(orarchitecture.md) — Is this behavior intentional by design? - Check
tech-stack.md— Does the framework handle this case? - Check
tech-stack.md## Accepted Patterns— Is this pattern explicitly documented as intentional? - Check
product.md— Is this actually a requirement violation? - Check existing tests — Is this behavior already tested and expected?
- Check
Framework/Library Verification
- Read official docs for the specific method/pattern in question
- Quote relevant doc section proving this is/isn't handled
- Check framework version in tech-stack.md (behavior may vary by version)
- Look for middleware, interceptors, or global handlers that may address the issue
Example Framework Documentation Quote:
"React automatically escapes JSX content to prevent XSS (React Docs: Main Concepts > JSX). However, dangerouslySetInnerHTML bypasses this protection. Framework version: React 18.2.0 (from tech-stack.md)."
Codebase Pattern Check
- Search for similar patterns elsewhere in codebase
- If pattern is used consistently, verify it's actually buggy (not just unfamiliar)
- Check if there's a project-specific utility/wrapper that handles the concern
False Positive Elimination
- Is this dead code that's never executed?
- Is this test/mock/stub code not in production?
- Is this intentionally disabled (feature flag, config)?
- Is there a comment explaining why this appears unsafe but is actually safe?
Pattern Prevalence Check (before reporting)
- Run Grep to find all occurrences of the pattern
- If found >5x:
- Randomly sample 3 instances
- Verify they exhibit the same suspected bug
- If they work correctly, investigate: what's different about THIS instance?
- If no difference found and other instances work: DO NOT REPORT
- If all instances have the bug: Report with pattern count in "Impact"
Example Pattern Prevalence Check:
1. Grep: `rg 'dangerouslySetInnerHTML' src/` → found 12 occurrences
2. Sampled 3: src/Blog.tsx:45, src/About.tsx:12, src/FAQ.tsx:30
3. All 3 sanitize input via `DOMPurify.sanitize()` before rendering
4. THIS instance (src/Comment.tsx:88) passes raw user input without sanitization
5. Decision: REPORT — this instance lacks the sanitization all others have
Confidence Levels
Only report bugs with HIGH or CONFIRMED confidence:
| Level | Criteria | Action |
|---|---|---|
| CONFIRMED | Verified through code trace, no mitigating factors found | Report as bug |
| HIGH | Strong evidence, checked context, no obvious mitigation | Report as bug |
| MEDIUM | Suspicious but couldn't verify all factors | Ask user to confirm before reporting |
| LOW | Possible issue but likely handled elsewhere | Do NOT report |
Example confirmation prompt for MEDIUM Confidence:
"I found a potential race condition in src/handler.ts:45 where async state updates may overwrite each other. However, I couldn't verify if there's a locking mechanism elsewhere. Should I report this as a bug?"
Evidence Requirements
Each reported bug MUST include:
- Code Evidence: The actual problematic code snippet
- Trace: How data reaches this point (caller chain or data flow)
- Verification Done: Which checks from the checklist were completed
- Why Not a False Positive: Explicit statement of why this isn't handled elsewhere
Analysis Rules
- Do not execute code - Reason from source only
- Do not assume frameworks "handle it" - Verify explicitly by checking docs/code
- Do not assume code is buggy - Verify it's actually reachable and unguarded
- Trace data flow completely - From input source to bug location
- Cross-reference ALL Draft context - Check architecture, tech-stack, product, tests
- Check for existing mitigations - Middleware, wrappers, utilities, global handlers
- Search for patterns - If used elsewhere without issues, investigate why
Optional: Runtime Verification (if test suite exists)
For suspected bugs that can be tested, write a minimal failing test to confirm:
- Write minimal test — Target the specific bug, not the entire feature
- Run test — Execute and observe failure
- Confirm bug — If test fails as predicted, confidence level increases to CONFIRMED
- Only report if: Test fails OR CONFIRMED confidence from code trace
Example:
// Suspected bug: off-by-one in pagination
test('should handle last page boundary', () => {
const items = Array(100).fill('item');
const result = paginate(items, { page: 10, perPage: 10 });
expect(result.items.length).toBe(10); // Currently returns 9
});
If test fails, upgrade confidence to CONFIRMED and include test in bug report.
Regression Test Generation
For each verified bug, generate a regression test in the project's native test framework that would expose the bug as a failing test. Before writing any new test, first discover the project's language/framework and whether existing tests already cover (or partially cover) the bug scenario.
Step 1: Detect Language & Test Framework
Identify the project's language(s) and test framework by examining the codebase:
| Signal | Language | Test Framework | Build/Run Command |
|---|---|---|---|
BUILD/WORKSPACE/MODULE.bazel + .cpp/.cc/.h |
C/C++ | GTest | bazel build / bazel test |
CMakeLists.txt + .cpp/.cc |
C/C++ | GTest | cmake --build / ctest |
go.mod or go.sum |
Go | testing (stdlib) |
go test |
pytest.ini/pyproject.toml/setup.py/conftest.py |
Python | pytest | pytest |
requirements.txt + unittest imports |
Python | unittest | python -m pytest |
package.json + Jest config |
JavaScript/TypeScript | Jest | npx jest / npm test |
package.json + Vitest config |
JavaScript/TypeScript | Vitest | npx vitest |
package.json + Mocha config |
JavaScript/TypeScript | Mocha | npx mocha |
Cargo.toml |
Rust | built-in #[test] |
cargo test |
pom.xml |
Java | JUnit | mvn test |
build.gradle/build.gradle.kts |
Java/Kotlin | JUnit | gradle test |
Resolution order:
- Check
draft/tech-stack.mdfirst — it may explicitly state the test framework - Look for existing test files and match their import/framework patterns
- Fall back to build system signals above
If the project is polyglot (multiple languages), detect per-component and generate tests in the matching language for each bug.
If no test framework is detected: Mark all bugs with Regression Test Status: N/A — no test framework detected and proceed with bug reporting. Do not skip bugs because tests cannot be written. The regression test section is supplementary — the primary deliverable is the bug report.
Record the detected configuration:
Language: [detected | none]
Test Framework: [detected | none]
Build System: [detected | none]
Test Command: [detected | N/A]
Step 2: Existing Test Discovery (REQUIRED per bug, skip if no test framework)
For each verified bug, search the codebase for existing tests before generating new ones:
Locate test files for the buggy module using language-appropriate patterns:
Language Search Patterns C/C++ *_test.cpp,*_test.cc,test_*.cpp; patterns:TEST(,TEST_F(,TEST_P(Go *_test.goin same package; patterns:func Test,func BenchmarkPython test_*.py,*_test.pyintests/; patterns:def test_,class TestJS/TS *.test.ts,*.spec.ts,__tests__/*.ts; patterns:describe(,it(,test(Rust #[cfg(test)]in same file, ortests/*.rs; patterns:#[test],fn test_Java *Test.java,*Tests.javainsrc/test/; patterns:@Test,@ParameterizedTestAnalyze existing test coverage
- Read each related test file found
- Check if any test exercises the exact code path that triggers the bug
- Check if any test covers the same function/method but misses the specific edge case
- Check if a test exists but has a wrong assertion (asserts buggy behavior as correct)
Classify the coverage status — one of:
Status Meaning Action COVERED Existing test already catches this bug (test fails on buggy code) Report the existing test — no new test needed PARTIAL Test exists for the function but misses this specific scenario Add the missing case to the existing test file WRONG_ASSERTION Test exists but asserts the buggy behavior as correct Fix the assertion in the existing test NO_COVERAGE No test exists for this code path Generate a new test N/A Bug is in non-testable code (config, markdown, LLM workflow) Write N/A — [reason]Document discovery results in the bug report's Regression Test field
Example Existing Test Discovery:
1. Bug location: src/parser.cpp:145 — off-by-one in tokenize()
2. Grep: `rg 'tokenize' tests/` → found tests/parser_test.cpp
3. Read tests/parser_test.cpp:
- TEST(Parser, TokenizeSimpleInput) — tests basic input ✓
- TEST(Parser, TokenizeEmptyString) — tests empty string ✓
- No test for boundary input length (the bug trigger)
4. Status: PARTIAL — parser_test.cpp covers tokenize() but misses boundary case
5. Action: Add new TEST case to existing tests/parser_test.cpp
Step 3: Generate or Modify Test Cases
Based on discovery results, generate tests in the project's native framework:
When status is COVERED
**Regression Test:**
**Status:** COVERED — existing test already catches this bug
**Existing Test:** `tests/parser_test.cpp:45` — `TEST(Parser, TokenizeBoundary)`
No new test needed.
When status is PARTIAL — add to existing test file
When status is WRONG_ASSERTION — fix assertion in existing test
When status is NO_COVERAGE — generate new test
Test Case Requirements (all languages)
Each new test MUST:
- Target exactly one bug — One test per finding, named after the bug
- Use descriptive test names — Language-idiomatic naming (see templates below)
- Include the bug setup — Reproduce the preconditions that trigger the bug
- Assert the expected (correct) behavior — The test should FAIL against the current buggy code
- Comment the expected vs actual — Explain what the test expects and what currently happens
- Be self-contained — Include necessary imports, minimal fixtures, no external dependencies beyond the test framework and project modules
- Specify target file — State whether this goes in an existing test file or a new one
Language-Specific Test Templates
C/C++ (GTest)
#include <gtest/gtest.h>
// #include "relevant/project/header.h"
// Bug: [SEVERITY] Category: Brief Title
// Location: path/to/file.cpp:line
// This test FAILS against current code, PASSES after fix
TEST(BugCategory, BriefBugTitle) {
// Setup
// Act
// Assert
EXPECT_EQ(actual, expected) << "Description of what should happen";
}
Python (pytest)
# Bug: [SEVERITY] Category: Brief Title
# Location: path/to/file.py:line
# This test FAILS against current code, PASSES after fix
import pytest
from module.under.test import function_under_test
def test_brief_bug_title():
"""[Category] Brief description of the bug scenario."""
# Setup
# Act
result = function_under_test(input)
# Assert
assert result == expected, "Description of what should happen"
Go (testing)
package package_name
import (
"testing"
// project imports
)
// Bug: [SEVERITY] Category: Brief Title
// Location: path/to/file.go:line
// This test FAILS against current code, PASSES after fix
func TestBriefBugTitle(t *testing.T) {
// Setup
// Act
got := FunctionUnderTest(input)
// Assert
if got != expected {
t.Errorf("FunctionUnderTest() = %v, want %v", got, expected)
}
}
JavaScript/TypeScript (Jest/Vitest)
// Bug: [SEVERITY] Category: Brief Title
// Location: path/to/file.ts:line
// This test FAILS against current code, PASSES after fix
import { functionUnderTest } from './module-under-test';
describe('BugCategory', () => {
it('should brief bug title', () => {
// Setup
// Act
const result = functionUnderTest(input);
// Assert
expect(result).toBe(expected);
});
});
Rust (#[test])
// Bug: [SEVERITY] Category: Brief Title
// Location: path/to/file.rs:line
// This test FAILS against current code, PASSES after fix
#[cfg(test)]
mod bug_regression_tests {
use super::*;
#[test]
fn test_brief_bug_title() {
// Setup
// Act
let result = function_under_test(input);
// Assert
assert_eq!(result, expected, "Description of what should happen");
}
}
Java (JUnit 5)
// Bug: [SEVERITY] Category: Brief Title
// Location: path/to/File.java:line
// This test FAILS against current code, PASSES after fix
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
class BugCategoryTest {
@Test
void briefBugTitle() {
// Setup
// Act
var result = classUnderTest.methodUnderTest(input);
// Assert
assertEquals(expected, result, "Description of what should happen");
}
}
Consolidated Test File
After all bugs are documented, collect all test cases into a single consolidated section in the report (see Report Generation). Group by discovery status so the reader knows which tests are new vs modifications to existing tests.
Step 4: Test Infrastructure Discovery
Before writing any test files, discover the project's test infrastructure and conventions:
Detect Build System & Test Runner
Language Build System Signals Test Runner C/C++ WORKSPACE/MODULE.bazel→ Bazel;CMakeLists.txt→ CMakebazel test/ctestGo go.mod(always present)go test ./...Python pyproject.toml/setup.cfg/tox.ini/ barepytest(prefer) /python -m unittestJS/TS package.json→ checkscripts.testand devDepsnpx jest/npx vitest/npm testRust Cargo.toml(always present)cargo testJava pom.xml→ Maven;build.gradle→ Gradlemvn test/gradle testIf no recognized build system is found, inform user and keep report-only test output:
"No recognized build/test system detected. Regression tests are included in the report only."Map Source Files to Test Locations For each buggy source file, determine where its tests live (or should live):
Language Common Conventions C/C++ (Bazel) Co-located foo_test.cppor separatetests/tree; checkcc_testin BUILDGo Same directory: foo.go→foo_test.go(always co-located)Python src/auth/handler.py→tests/auth/test_handler.pyortests/test_auth_handler.pyJS/TS src/auth/handler.ts→src/auth/handler.test.tsor__tests__/handler.test.tsRust In-file #[cfg(test)]module, ortests/directory for integration testsJava src/main/java/com/...→src/test/java/com/...(Maven convention)- If tests exist: record the directory, naming convention, and any build config
- If no tests exist: adopt the project's dominant convention
- If no convention exists: default to a
tests/directory mirroring the source tree
Identify Test Dependencies (language-specific)
Language What to Find C/C++ (Bazel) GTest dep label: @com_google_googletest//:gtest_main; sourcecc_librarytargetsGo No extra deps needed ( testingis stdlib)Python Check if pytestis inrequirements*.txt/pyproject.toml; add if missingJS/TS Check if test framework is in devDependencies; identify import styleRust No extra deps for unit tests; dev-dependenciesfor integration test cratesJava JUnit version in pom.xml/build.gradledependencies
Step 5: Write Test Files (only for testable bugs)
Skip this step entirely if no test framework was detected in Step 1.
For bugs with status NO_COVERAGE, PARTIAL, or WRONG_ASSERTION, write the actual test files. Bugs with COVERED or N/A status do not need action here — they are still included in the final report:
NO_COVERAGE — Create new test file
Create directory if it doesn't exist:
mkdir -p <test_directory>/Write the test file using the language-appropriate template:
Language Example Target File C/C++ tests/auth/login_handler_test.cppGo auth/login_handler_test.go(same package)Python tests/auth/test_login_handler.pyJS/TS src/auth/login_handler.test.tsor__tests__/auth/login_handler.test.tsRust tests/login_handler_test.rsor#[cfg(test)]in sourceJava src/test/java/com/example/auth/LoginHandlerTest.javaCreate or update build config (if required by the build system):
C/C++ (Bazel) — add
cc_testto BUILD:cc_test( name = "<source_filename>_test", srcs = ["<source_filename>_test.cpp"], deps = [ "//src/<component>:<library_target>", "@com_google_googletest//:gtest_main", ], )Java (Maven) — no build config change needed (convention-based discovery) Java (Gradle) — no build config change needed Go — no build config change needed (
go testdiscovers_test.goautomatically) Python — no build config change needed (pytestdiscoverstest_*.pyautomatically) JS/TS — no build config change needed (Jest/Vitest discover*.test.*automatically) Rust — no build config change needed (cargo testdiscovers#[test]automatically)If multiple bugs affect different files in the same component, create one test file per source file (not one per bug). Group related bug tests into the same file.
PARTIAL — Add test case to existing file
- Read the existing test file
- Append the new test at the idiomatic location:
- C/C++: Before closing namespace brace
- Go: End of file (same package)
- Python: End of file or within existing test class
- JS/TS: Inside the relevant
describe()block, or at end of file - Rust: Inside existing
#[cfg(test)]module - Java: Inside existing test class, before closing brace
- No build config changes needed
WRONG_ASSERTION — Fix assertion in existing file
- Read the existing test file
- Locate the wrong assertion
- Replace with the corrected assertion
- No build config changes needed
Constraints:
- Never modify production source code — only test files and their build configs
- Each test file must be valid for the project's test runner
- Use the project's actual import paths, module names, and namespace conventions
- Match existing test style (fixtures, helpers, naming conventions)
Step 6: Build & Syntax Validation
After writing all test files, validate them using the project's native toolchain.
Validate each new/modified test using the language-appropriate command:
Language Validation Command What It Checks C/C++ (Bazel) bazel build //tests/<component>:<target>_testCompilation + linking C/C++ (CMake) cmake --build <build_dir> --target <target>_testCompilation + linking Go go vet ./path/to/package/...Syntax + type checking (no execution) Python python -m py_compile tests/path/test_file.pySyntax validation JS/TS npx tsc --noEmit tests/path/file.test.ts(TS) ornode --check tests/path/file.test.js(JS)Type check / syntax Rust cargo check --testsType check + borrow check (no execution) Java (Maven) mvn test-compileCompilation only Java (Gradle) gradle testClassesCompilation only Handle validation results:
Result Action Succeeds Mark as BUILD_OKin reportFails — import/include error Fix the import path, retry (up to 2 retries) Fails — missing dep Add the dependency, retry (up to 2 retries) Fails — type/API mismatch Fix the test to match actual API signatures, retry (up to 2 retries) Persistent failure (3 attempts) Mark as BUILD_FAILEDwith the error message in report. Delete the broken test file and note in the report: "Test file removed due to persistent build failure."Do NOT run the tests. The tests are designed to FAIL against the current buggy code — that's the point. Validation checks only syntax, types, and linking. Running them would produce expected failures that aren't useful here.
Exception for Go:
go vetis preferred overgo buildfor test files because Go compiles tests as part ofgo testonly.go vetcatches type errors and common issues without executing.Validation summary — Record results for the report:
BUILD_OK: 3 targets BUILD_FAILED: 1 target (tests/config/test_loader.py — ImportError: no module named 'config.loader') SKIPPED: 1 target (N/A — race condition not reliably testable)
Fix Suggestion Generation
For each bug with CONFIRMED or HIGH confidence, generate a minimal suggested fix alongside the bug report. Fix suggestions are advisory — they are never auto-applied.
Fix Generation Rules
- Minimal change principle: The fix must be the smallest code change that addresses the root cause. Do not refactor surrounding code, add features, or improve style.
- Before/after format: Include the exact current code (BEFORE) and the suggested fix (AFTER) as code snippets with file path and line numbers.
- Root cause targeting: The fix must address the root cause identified in the bug's data flow trace, not a symptom. If the root cause is in a different location than the symptom, fix at the root.
- Mark as SUGGESTED: Every fix must be clearly marked as
SUGGESTED (REVIEW REQUIRED)— never imply auto-application. - One fix per bug: Each bug gets exactly one suggested fix. If multiple fix strategies exist, choose the most conservative one and note alternatives.
- Preserve behavior: The fix must not change behavior beyond correcting the identified bug. No side-effect improvements.
- Skip when inappropriate: Mark fix as
N/Afor bugs where the fix requires architectural changes, significant refactoring, or domain knowledge beyond what the code provides.
Reference: Meta SapFix — automated fix suggestion with human-in-the-loop validation.
Output Format
For each verified bug:
### [SEVERITY] Category: Brief Title
**Location:** `path/to/file.ts:123`
**Confidence:** [CONFIRMED | HIGH | MEDIUM]
**Code Evidence:**
```[language]
// The actual problematic code
Data Flow Trace: [How data reaches this point: caller → caller → this function]
Issue: [Precise technical description of what is wrong]
Impact: [User-visible effect or system failure mode]
Verification Done:
- Traced code path from [entry point]
- Checked architecture.md — not intentional
- Verified framework doesn't handle this
- No upstream guards found in [files checked]
Why Not a False Positive: [Explicit statement: "No sanitization exists because X", "Framework Y doesn't escape Z in this context", etc.]
Fix: [Minimal code change or mitigation]
Suggested Fix (REVIEW REQUIRED):
// BEFORE (current buggy code):
[exact code snippet from the codebase]
// AFTER (suggested fix):
[minimal change that addresses root cause]
This fix is SUGGESTED only — human review required before applying. Reference: Meta SapFix methodology.
Regression Test:
Status: [COVERED | PARTIAL | WRONG_ASSERTION | NO_COVERAGE | N/A]
Existing Test: [path/to/test_file:line — test name | None found]
[Action: existing test reference, proposed modification, or new test case]
// New or modified test case (omit if COVERED or N/A)
**Example — COVERED (no new test needed):**
```markdown
**Regression Test:**
**Status:** COVERED — existing test already catches this bug
**Existing Test:** `tests/validator_test.cpp:89` — `TEST(Validator, RejectsScriptTags)`
No new test needed. Existing test fails when XSS sanitization is removed.
Example — PARTIAL (C++ / GTest):
**Regression Test:**
**Status:** PARTIAL — tests exist for processInput() but miss unsanitized HTML path
**Existing Test File:** `tests/input_test.cpp`
**Modification:** Add to existing file:
```cpp
TEST(InputSanitization, RejectsMaliciousScript) {
std::string malicious = "<script>alert('xss')</script>";
std::string result = processInput(malicious);
EXPECT_EQ(result.find("<script>"), std::string::npos)
<< "Input should be sanitized to remove script tags";
}
**Example — NO_COVERAGE (Python / pytest):**
```markdown
**Regression Test:**
**Status:** NO_COVERAGE — no tests found for process_input()
**Target File:** `tests/test_input_processor.py` (new file)
```python
import pytest
from input.processor import process_input
def test_rejects_malicious_script():
"""Input should be sanitized to remove script tags."""
malicious = "<script>alert('xss')</script>"
result = process_input(malicious)
assert "<script>" not in result, "XSS script tag should be stripped"
# Expected: FAILS against current code (passes XSS through), PASSES after fix
**Example — NO_COVERAGE (Go / testing):**
```markdown
**Regression Test:**
**Status:** NO_COVERAGE — no tests found for ProcessInput()
**Target File:** `input/processor_test.go` (new file)
```go
package input
import (
"strings"
"testing"
)
func TestProcessInputRejectsMaliciousScript(t *testing.T) {
malicious := "<script>alert('xss')</script>"
result := ProcessInput(malicious)
if strings.Contains(result, "<script>") {
t.Error("XSS script tag should be stripped from input")
}
}
// Expected: FAILS against current code (passes XSS through), PASSES after fix
**Example — N/A (not testable, but still report the bug):**
```markdown
**Regression Test:**
**Status:** N/A — environment config, no executable code path
**Reason:** Bug is in `config/production.yaml` which sets incorrect timeout value. Config files are not unit-testable; fix requires changin
…(truncated)