Loop Tester

Loop Tester

Treat testing as a resumable campaign, not a finite checklist. Repeatedly choose the highest-information risk, perform real operations, verify outcomes through independent observations, preserve evidence, checkpoint state, and continue until a valid stop condition occurs.

Honor the continuation contract

• Continue after a batch or round completes. Immediately select and execute the next useful batch.
• Do not stop because all planned cases ran, no new defect appeared, coverage reads 100%, or one path is blocked.
• Increase depth between rounds by changing inputs, ordering, roles, state, scale, environment, concurrency, or module combinations. Repeat an identical case only for reproduction, regression, stability, or change verification.
• Use the host's persistent goal, continuation, wakeup, or monitoring mechanism when one is available and the user's request authorizes long-running execution. Keep the goal active while safe work remains.
• Never simulate persistence with an uncontrolled shell loop, and never claim to be running in the background when the host cannot continue execution. Before any unavoidable execution boundary, write a complete checkpoint that allows the next invocation to resume exactly.
• Keep one bounded batch in progress at a time. A batch should produce observable evidence and a checkpoint before more work is queued.

Establish scope and safety

1. Read repository instructions, the current worktree state, start commands, existing tests, configuration, known issues, and recent changes.
2. Resolve the target, authorization boundary, environment type, accounts or roles, permitted test data, external dependencies, and destructive-action limits from available context.
3. Default to a local, development, staging, or disposable environment. Obtain explicit authority before touching production, real user data, paid services, destructive flows, load generation, fault injection, or third-party systems.
4. Preserve unrelated user changes. Test and record by default; do not modify product code unless the user separately asks for a fix.
5. Prefer isolated test accounts, temporary data, dedicated worktrees, and reversible operations. Define cleanup before executing a case with side effects.

Do not abandon the whole campaign when one prerequisite is unknown. Record the narrow blockage and continue any safe, independent work. Ask the user only when the missing decision would materially change scope, authorization, or risk.

Initialize or resume the campaign

Determine this skill's directory, then run:

python3 <skill-dir>/scripts/init_testing_loop.py <project-root>
python3 <skill-dir>/scripts/validate_testing_loop.py <project-root>

The initializer only creates missing state and never overwrites history. Read state-contract.md before repairing state or writing the first checkpoint.

Use .testing-loop/CURRENT_STATE.md as the authoritative recovery pointer. On every invocation:

1. Read CURRENT_STATE.md, PROGRESS.md, REGRESSION_QUEUE.md, and BLOCKED_ITEMS.md.
2. Read the relevant portions of TEST_PLAN.md, COVERAGE.md, FINDINGS.md, and recent TEST_HISTORY.md entries.
3. Compare the recorded code version and environment with the current ones. Treat changes as new regression signals.
4. Resume the recorded next action unless new evidence makes a different action higher priority.
5. Build or update the test map: start paths, routes, modules, roles, forms, APIs, persistence, integrations, complex interactions, and observability surfaces.
6. Start only the services needed for the current batch. Record commands, ports, process ownership, accounts, version, and environment differences.

If UI automation is unavailable, continue with safe API, CLI, log, or static test-map work and record UI-only cases as blocked. Do not mark them covered.

Execute the closed loop

Repeat this sequence without waiting for another user prompt:

1. Read state — inspect recent history, code changes, low-confidence coverage, due regressions, open findings, recovered blockages, and the last confirmed state.
2. Select risk — choose the candidate expected to add the most information. Use the scoring and novelty rules in test-catalog.md when the next target is not obvious.
3. Design a batch — define preconditions, role, data, steps, observations, oracle, cleanup, resource budget, and the condition for ending this case.
4. Perform real operations — interact through the same UI, API, CLI, or service boundary a user would use. Avoid replacing manual interaction with source inspection alone.
5. Observe broadly — capture visible behavior, requests and responses, console output, service logs, persisted state, timing, and resource behavior as applicable.
6. Verify independently — do not trust a toast or status code alone. Cross-check at least one independent surface, such as refresh, list versus detail, API versus UI, database versus API, relogin, or a second role.
7. Triage anomalies immediately — distinguish product defects from environment, fixture, automation, and expectation failures. Follow evidence-and-triage.md.
8. Persist results — save evidence first, update or create the finding, then update the index, coverage, history, queues, progress, and finally CURRENT_STATE.md.
9. Clean up — remove only disposable data and stop only resources started by this campaign that are no longer needed. Preserve the state required for reproduction.
10. Generate and start the next batch — use the new evidence to reprioritize; do not merely rerun the previous steps.

Rotate through progressive strategies

Start with broad baseline coverage, then deepen through input boundaries, abnormal recovery, cross-module and concurrency combinations, authorization and isolation, stability and resource behavior, and finding-directed exploration. From the eighth round onward, choose dynamically from coverage gaps, recent changes, defect clusters, long-unchecked regressions, recovered blockages, complex areas, and missing evidence.

Treat these as strategy families rather than a rigid eight-round script. When a critical flow is untested or a fresh change lands, return to the relevant earlier strategy with new state and evidence. Read test-catalog.md before generating a new strategy or when two consecutive batches add no new information.

Record findings and evidence

• Create one file per independent issue under .testing-loop/findings/FINDING-NNNN.md. To allocate an ID and scaffold the record safely, run:

  python3 <skill-dir>/scripts/create_finding.py <project-root> \
    --title "<title>" --module "<module>" --severity "<severity>" --kind "<type>"
  

• Record observed facts immediately; add missing detail later. Label hypotheses as hypotheses and never invent unavailable fields.

• Reproduce with the smallest reliable sequence, record frequency and impact, and test adjacent paths that may share the cause.

• Link every screenshot, request trace, log excerpt, data snapshot, and recording to a case or finding ID. Redact tokens, cookies, credentials, personal data, and unrelated customer content.

• Preserve closed findings and prior regression results. Change their lifecycle state; do not erase history.

• A visual success message is not proof of persistence, and a client failure is not proof that the server performed no side effect.

Handle blockage and recovery

For a localized blockage:

1. Save the last confirmed state and the attempted action.
2. Gather enough evidence to classify the blockage.
3. Try a small number of reversible recovery actions, such as one retry, restarting a service started by this campaign, recreating disposable data, or switching entry points.
4. Record the blockage, attempts, unlock condition, owner if known, and next review time in BLOCKED_ITEMS.md.
5. Move to another testable risk and revisit the item when its condition changes or its review is due.

After a crash, service exit, or browser disconnect, checkpoint evidence before restarting. Resume from the last verified state and avoid replaying a possibly committed operation until its side-effect status is known.

If the same campaign-wide blocking condition persists and no safe work remains, follow the host's exact persistent-goal blocking contract. When that contract counts goal turns, require three consecutive goal turns with the same blocker; multiple checks or batches inside one turn do not satisfy it. Without a host-defined threshold, require three separate resume attempts before writing a final blocked checkpoint. Do not use this condition for ordinary local failures.

Communicate and checkpoint

• During active work, send a concise progress update at least every 60 seconds: current module, risk under test, latest evidence or finding, and next action.

• When asked for status, report verifiable counts and artifacts first, then continue unless the user asks to pause or stop.

• Update CURRENT_STATE.md after every batch, module switch, crash, environment change, and execution boundary.

• End each round by recording scope, new coverage, findings, reproduction state, weak areas, and the next-round plan; then begin that plan.

• Run the validator after initialization, state repair, and major checkpoints:

  python3 <skill-dir>/scripts/validate_testing_loop.py <project-root>
  

Repair validation errors without deleting history.

Stop and hand off

Stop only when:

• the user explicitly asks to stop, pause, exit, or terminate;
• the host externally terminates execution; or
• all available environments remain unrecoverable after the campaign-wide blocked audit.

Before stopping, update all state files, validate them, close resources started by the campaign, and hand off the current conclusion, open findings, unfinished batch, blocked work, exact resume action, and environment restoration steps.