QA Check — Execute Test Cases
Runs the test-cases.md artifact produced by /aif-qa and records execution status in qa-check.md.
In agent mode, it also maintains reusable cross-QA execution memory under the QA root:
agent-context.md — curated, current, reusable non-sensitive setup facts for future automated QA runs
agent-history.md — append-only reusable learnings extracted from prior runs, not a per-run audit log
For browser/UI cases, agent mode saves executable replay scripts under the branch-specific browser-replay/ directory. Later runs execute those scripts first, including scripts for previously passed cases, so fixes are retested and prior behavior gets regression coverage without regenerating browser automation.
The skill is stack-free and agent-free: it does not assume a framework, package manager, browser tool name, or agent runtime. In agent mode, it uses the appropriate available execution surface for each case: live browser automation, CLI commands, project test runners, HTTP/API calls, database-safe read checks, file/document inspection, or other non-destructive automated verification.
Modes
| Argument |
Mode |
What you do |
human |
Human-guided QA |
Show exactly one test case, ask the user whether it works, and record their answer |
agent |
Automated-agent QA |
Execute test cases through the most appropriate available capability and record observed results |
If no mode is provided, ask the user which mode to run.
Workflow
Step 0: Load Config
FIRST: Read .ai-factory/config.yaml if it exists to resolve:
- Paths:
paths.description, paths.architecture, paths.qa (default: .ai-factory/qa)
- Language:
language.ui for AskUserQuestion prompts, progress messages, summaries, and next-step guidance
language.artifacts for the persisted qa-check.md artifact
language.technical_terms for human-readable technical terminology style in the artifact
- If
language.artifacts is missing, use language.ui
- If both are missing, use
en
- Git:
git.enabled for branch resolution
If config.yaml does not exist, use defaults:
- DESCRIPTION.md:
.ai-factory/DESCRIPTION.md
- ARCHITECTURE.md:
.ai-factory/ARCHITECTURE.md
- QA artifacts:
.ai-factory/qa/
ui_language: en
artifact_language: en
technical_terms_policy: keep
- Git enabled:
true
Store:
ui_language = language.ui || "en"
artifact_language = language.artifacts || language.ui || "en"
technical_terms_policy = language.technical_terms || "keep"
git_enabled = git.enabled when present, otherwise true
qa_root = resolved paths.qa || ".ai-factory/qa/"
qa_agent_context_path = <qa_root>/agent-context.md
qa_agent_history_path = <qa_root>/agent-history.md
All AskUserQuestion prompts, user-visible explanations, per-case summaries, and final summaries MUST be written in ui_language.
The persisted qa-check.md artifact MUST be written in artifact_language.
Templates define structure, not language. Use the canonical English template in templates/QA-CHECK.md. If artifact_language is not en, translate headings, labels, status text, comments you author, and explanatory text to artifact_language before saving. Preserve checkbox syntax, test case IDs (TC-001), commands, paths, config keys, URLs, selectors, package names, API names, branch names, and raw error messages.
For artifact_language = ru, write human-readable prose, headings, statuses, summaries, and agent-authored comments in Russian. Preserve user wording except mandatory redaction of sensitive values before writing.
Apply technical_terms_policy while writing artifacts:
keep — keep common technical terms such as browser, selector, viewport, endpoint, payload, regression, and fixture when clearer
translate — translate human-readable technical terms where a natural target-language term exists
mixed — translate ordinary prose terms while keeping code, infrastructure, and ecosystem terms unchanged
Step 0.1: Load Project Context
Read the resolved description path if it exists.
Read the resolved architecture path if it exists.
Read .ai-factory/skill-context/aif-qa-check/SKILL.md — MANDATORY if the file exists. Treat it as project-level overrides for this skill.
Read <paths.qa>/agent-context.md if it exists. Treat it as reusable, user-approved QA execution context for future automated runs: stable environment classes, canonical local/staging URLs, login route, test account role, allowed non-sensitive usernames, required seed data patterns, stable selectors, field identifiers, known redirects, startup prerequisites, safe command patterns, reusable test-filter conventions, environment notes, and service startup instructions.
Read <paths.qa>/agent-history.md if it exists. Treat it as append-only cross-QA memory from prior /aif-qa-check runs. Before asking the user or attempting automated execution, search it for reusable prior answers to the same blocker so the skill does not repeat avoidable mistakes.
agent-context.md and agent-history.md are global to all QA sets under paths.qa. They MUST NOT become logs for a specific QA plan. Do not write branch names, branch slugs, QA target paths, current artifact_dir, TC-* mappings, per-run summary counts, exhaustive command transcripts, assertion counts from a single run, one-off file checks, or plan-specific labels such as a numbered QA folder name. Write those details only to <paths.qa>/<branch-slug>/qa-check.md.
Only promote information into agent-context.md or agent-history.md when it is likely to help future unrelated QA runs. Prefer stable patterns over concrete one-off run facts. Example: Laravel backend checks use php artisan test --filter=<TestClass> is reusable; 05-profile-formula-recommendations ran ProfileReaderTest with 4 tests / 13 assertions is run-specific and belongs only in qa-check.md.
Never write global negative capability or scope claims unless they are true for the whole project. Do not write statements like browser automation is not required, browser URL is not used, all QA is backend/CLI, or these cases are backend-test/cli/file-docs to agent-context.md or agent-history.md when that conclusion came from one QA plan. Instead, keep it branch-specific in qa-check.md, or phrase a reusable conditional rule such as For backend-only QA cases, browser automation is not required; browser/UI cases still require Browser or Playwright MCP.
Never treat either file as a source of secrets. If either file contains an apparent password, token, cookie, authorization header, one-time code, or other secret, ignore that value for execution and redact it the next time the file is rewritten or appended to.
Step 0.2: Parse Arguments and Resolve Branch
Parse $ARGUMENTS:
- Detect mode: first word matching
human or agent; remove it from arguments.
- Detect branch: remaining text, if any, is the branch label.
- If no mode was provided, ask in
ui_language which mode to run:
- Human-guided QA
- Automated-agent QA
Resolve the working branch:
If git_enabled = false or the repository is not a git work tree:
If branch was provided in arguments → use it as the resolved branch label
Otherwise → set resolved_branch = "manual"
If git_enabled = true and the repository is a git work tree:
If branch was provided in arguments → use it as the resolved branch
Otherwise → run: git branch --show-current
Store:
resolved_branch
artifact_dir = <resolved paths.qa>/<branch-slug>
test_cases_path = <artifact_dir>/test-cases.md
qa_check_path = <artifact_dir>/qa-check.md
browser_replay_dir = <artifact_dir>/browser-replay
Compute branch-slug with the exact same algorithm as /aif-qa:
- Replace every character not in
[A-Za-z0-9._-] with -, collapse consecutive -, trim leading/trailing -, and use branch if empty. Then MUST truncate safe_slug to the first 40 ASCII characters. Because the normalized safe_slug alphabet is [A-Za-z0-9._-], byte length and character length are identical.
- Run
git hash-object --stdin <<< "<resolved_branch>" and take the first 8 hex characters as hash8.
- Combine:
branch-slug = "<safe_slug>-<hash8>".
Step 1: Verify QA Inputs
Check for <artifact_dir>/test-cases.md.
If it is missing, STOP and ask in ui_language whether to:
- Run
/aif-qa test-cases <resolved_branch> first
- Cancel
Read test-cases.md. Extract test cases by IDs (TC-001, TC-002, etc.), titles, priority, type, execution surface when present, preconditions, steps, expected results, and test data. Preserve their order.
Compute source binding metadata before creating or modifying qa-check.md:
source_digest — deterministic digest of the full test-cases.md content using git hash-object --no-filters <test_cases_path> when available, or git hash-object --stdin over the exact file content.
case_digests — deterministic per-case digest for each extracted TC-NNN, computed from that case's canonical block including title, priority, type, preconditions, steps, expected result, and test data.
tested_revision — when git_enabled = true and the repository is a git work tree, run git rev-parse HEAD and record the resolved commit SHA.
worktree_digest — when git_enabled = true and the repository is a git work tree, record a deterministic digest of the current working tree state so dirty-tree QA cannot be reused after local changes without a commit.
manual_build_id — when git_enabled = false or the repository is not a git work tree, ask the user for an explicit build/version identifier before creating or resuming results. Do not accept an empty identifier.
replay_script_digests — for each existing canonical <browser_replay_dir>/TC-NNN.js, hash the exact file content with git hash-object --no-filters <path> when available, or another stable SHA-1/SHA-256 digest. This binds evidence to the automation that actually ran even though QA-owned replay files are excluded from worktree_digest.
Canonicalize each per-case digest input exactly:
- Extract the raw Markdown block from the line containing the case's
TC-NNN identifier through the line before the next TC-NNN block or end of file.
- Normalize line endings from CRLF or CR to LF.
- Remove trailing spaces and tabs from each line.
- Trim leading and trailing blank lines from the extracted block.
- Preserve the original field order, bullet markers, indentation, internal blank lines, Markdown punctuation, and all remaining whitespace. Do not sort fields, collapse whitespace, or rewrite bullets.
- Wrap the normalized block with raw block boundaries before hashing:
BEGIN TC-NNN\n<normalized block>\nEND TC-NNN\n.
- Hash the wrapped block with
git hash-object --stdin when available, or another stable SHA-1/SHA-256 digest if git is unavailable.
Compute worktree_digest exactly when git is enabled and a git work tree exists:
- Capture
git status --porcelain=v1 --untracked-files=all.
- Capture
git diff --binary HEAD --.
- Exclude
qa_check_path and every file under browser_replay_dir from the status, diff, and untracked-file digest inputs so QA-owned result/replay artifacts do not stale their own results. Do not exclude test_cases_path; source changes are also tracked by source_digest.
- For each remaining untracked file listed by porcelain status, append
UNTRACKED <path> <content-digest> where <content-digest> is git hash-object --no-filters <path> when the file is readable.
- Normalize line endings in the combined input to LF and hash it with
git hash-object --stdin.
- If the filtered work tree input is clean, record the digest of the canonical string
clean\n.
If mode = agent, perform Step 1.1 before creating or modifying qa-check.md. Existing qa-check.md may be inspected read-only during this gate.
If qa-check.md exists, read it and resume from existing statuses only after comparing stored binding metadata to the current binding metadata:
- If
tested_revision changed, mark every prior result as Stale and unchecked, preserve prior comments/evidence as historical context, and require retest. Do not count stale pass/fail/block statuses as current.
- If
worktree_digest changed, mark every prior result as Stale and unchecked, preserve prior comments/evidence as historical context, and require retest. Do not count stale pass/fail/block statuses as current.
- If
manual_build_id changed, treat it the same as a tested revision change.
- If
source_digest changed, compare case_digests. Preserve current status only for cases whose per-case digest is unchanged and whose tested revision/manual build id and worktree digest are unchanged.
- If an existing case's digest changed, mark that case
Stale, unchecked, and require retest.
- If a new case appears, add it as unchecked
Pending.
- If a prior case no longer exists in
test-cases.md, keep its historical entry marked Stale or move it to an artifact-language "Stale / Removed Cases" section; never count it as current.
- Do not overwrite prior pass/fail comments unless the user explicitly chooses to retest that case.
- If a canonical replay file's actual content digest differs from the
Script digest recorded for that case, mark the case Stale, preserve the previous replay metadata/evidence, and require a proof run of the changed script. Missing legacy script-digest or target-fingerprint metadata means the script is unproven, not matching.
- Revision, worktree, or manual-build staleness alone does not invalidate a browser replay script. It remains reusable only while its embedded case digest, embedded target fingerprint, and actual script digest match the current case, approved target, and previously recorded script digest.
If qa-check.md does not exist, create it from templates/QA-CHECK.md using the extracted test cases and source binding metadata. Every case starts unchecked and Pending.
Step 1.1: Agent Mode Capability and Safety Preflight
Agent mode is user-only (disable-model-invocation: true) because it can perform live browser actions, shell commands, local service calls, and other checks with meaningful side effects.
Before executing any case or writing qa-check.md in agent mode:
- Detect available execution capabilities from the current runtime tools:
- shell/CLI command execution
- project test runners and build tools available through shell
- live browser automation through in-app Browser or Playwright MCP
- HTTP/API checks available through existing project tooling or safe shell commands
- file, document, and repository inspection through Read/Grep/Glob
- Classify each test case by the evidence surface it requires:
browser-ui, cli, backend-test, api, file-docs, database-read, hybrid, human, or unknown. If the case has an explicit Execution surface: field, use it unless the case text clearly contradicts it; record any override in agent-history.md.
- Choose the least invasive capability that can produce concrete evidence for that case. Browser automation is REQUIRED for browser/UI-observable cases and hybrid cases whose expected result depends on rendered web behavior. Do not substitute CLI commands, unit tests, static inspection, or backend checks for a browser/UI case; those may be recorded only as supporting evidence.
- If a case requires browser automation, prefer the in-app Browser capability if available. If in-app Browser is not available, use Playwright MCP if available.
- Prefer a browser capability that can execute a saved Playwright-compatible
page script when replay scripts exist.
- If a browser/UI case requires live browser automation and neither in-app Browser nor Playwright MCP is available, keep only that case unchecked, set status to
Blocked, ask the user in ui_language to enable a live browser capability, and append this blocker to <paths.qa>/agent-history.md. Continue with other cases that can be verified through CLI, tests, API, or file/document checks.
- Use
<paths.qa>/agent-context.md, <paths.qa>/agent-history.md, test-cases.md, change-summary.md, project docs, current browser state, and repository scripts to determine the target URL, service startup commands, test commands, and environment.
- For browser/UI cases that require authentication or specific user state, resolve a browser test identity before marking the case blocked:
- First search
agent-context.md, agent-history.md, test data, seeders/factories/fixtures, docs, local database-safe reads, and existing test users for a matching non-sensitive test identity.
- If the environment is clearly
local, test, or disposable development, and repository tooling provides a safe way to create or seed a synthetic test user/state, create the minimal disposable fixture needed for the case. Record the fixture identifier and setup command/evidence in qa-check.md.
- If creating a fixture requires a password, role, tenant, workspace, feature flag, seed command, or any unclear setup decision, ask the user before blocking. Ask whether to provide an existing test account or authorize creating a disposable fixture.
- If the user provides or authorizes disposable test credentials for a
local or test environment, use them for this run. Store credentials in agent-context.md only if the user explicitly says they are reusable test-only credentials and safe to persist. Otherwise store only the username/role and where to retrieve the secret.
- Never store production credentials, personal credentials, cookies, session tokens, one-time codes, or shared secrets in
agent-context.md or agent-history.md.
- If a user-suppliable prerequisite is missing, ask the user before blocking:
- application URL or login URL
- test account role or non-sensitive username
- where to find credentials without exposing them in artifacts
- authorization to create a disposable local/test browser fixture and the desired initial state
- required startup state, seed data, feature flag, tenant, organization, or workspace
- ambiguous login mechanism or authentication provider
- stable selector, field identifier, or page route needed to continue
- exact project command, test filter, service startup command, or safe API endpoint needed to continue
- Save reusable non-sensitive answers to
<paths.qa>/agent-context.md immediately only when the answer is general enough to help future QA sets. Append the redacted answer summary to <paths.qa>/agent-history.md only when it captures a reusable lesson; otherwise keep it in the branch-specific qa-check.md evidence/comment.
- Determine and display the target environment, including URL when applicable and inferred environment class:
local, development, staging, test, production, or unknown. For a browser target:
- Derive
normalized_base_url from the resolved target URL by removing user information, query, and fragment; preserving scheme, host, port, and deployment base path; and removing a trailing slash except for the origin root.
- Compute
target_fingerprint from the exact canonical input AIF TARGET\n<environment_class>\n<normalized_base_url>\n with git hash-object --stdin when available, or another stable SHA-1/SHA-256 digest.
- Record the environment class and
target_fingerprint in qa-check.md. Never treat a replay created for another target fingerprint as matching.
- Do not execute browser actions, API writes, CLI commands with side effects, database writes, destructive commands, payment, permission, email, notification, data export, or other external-side-effect steps against
production or unknown targets without explicit user authorization immediately before execution. Record the authorization decision and target class in qa-check.md. Append it to agent-history.md only when it creates a reusable cross-QA policy or recurring authorization lesson; do not persist broad production authorization for future runs.
- Inspect each case and proposed command for destructive, irreversible, payment, permission, email, notification, data export, database mutation, service restart, deployment, filesystem deletion, or other external-side-effect risk. Require explicit per-case authorization before executing any such case.
- Prefer read-only and test-scoped commands. Do not run destructive shell commands such as
rm, git reset, git checkout --, database resets, migrations against shared environments, deploy commands, or cleanup commands that delete user data unless the user explicitly authorizes the exact action.
- If authorization is denied or unclear, leave the case unchecked, set status to
Blocked, and write the blocker to qa-check.md without executing the risky action. Append the decision to agent-history.md only when it is a reusable cross-QA lesson, not merely a case-specific denial.
Redaction is mandatory for agent comments/evidence and all human-entered comments/evidence:
- Redact credentials, cookies, authorization values, session tokens, API keys, bearer tokens, basic auth values, one-time codes, and personal secrets.
- Redact sensitive URL parameters such as
token, access_token, refresh_token, id_token, code, secret, password, passwd, pwd, auth, key, api_key, session, sid, and jwt.
- Replace redacted values with
[REDACTED] before writing comments or evidence to qa-check.md.
- Do not paste raw browser storage, cookies, request headers, authorization headers, or token-bearing URLs into the artifact.
Step 2: Human Mode
Run exactly one pending or selected test case at a time.
For each case:
- Show only that test case's title, priority, preconditions, steps, test data, and expected result.
- Include a short per-case summary in
ui_language.
- The summary question MUST mean: "Test this and answer whether it works." For
ui_language = ru, use exactly: Протестируйте и ответьте работает или нет.
- Ask the user for the result:
- Works / completed
- Does not work / failed
- Stop for now
- If the user says it works, mark the case as checked (
[x]) in qa-check.md, set status to Passed, and add the current mode as human.
- If the user says it failed, keep the checkbox unchecked (
[ ]), set status to Failed, ask for the reason, and write the user's explanation as the comment while preserving user wording except mandatory redaction of sensitive values.
- Save
qa-check.md after every case so progress survives context resets.
Do not show the next test case until the current one has a result or the user stops.
Step 3: Agent Mode
Agent mode MUST produce concrete, case-appropriate evidence. Browser execution, CLI output, automated test output, API responses, backend command results, database-safe read results, or file/document inspection can each be enough to mark a case passed when that evidence directly covers the case steps and expected result.
Internal deterministic invariants MUST be treated as automated checks, not manual QA. Cases involving service-method calls, cache state, materialized rows, exact arrays, raw database values, formula outputs, CLI internals, or similar non-human-observable behavior should be classified as backend-test, cli, api, or database-read as appropriate. First look for existing tests or commands that already cover the invariant. If existing tests are found and pass, mark the corresponding TC-* case Passed with the test class/name, command/filter, exit status, and assertion/result summary as evidence. If one test run covers multiple TC-* cases, reuse that evidence on every covered case and optionally summarize the shared run under "Supporting Automated Checks". If no suitable automated coverage exists, keep the case Blocked with a blocker such as Missing automated test coverage for backend invariant; do not ask the user to manually verify internal arrays or raw database values.
Determine the test target:
- If
agent-context.md, agent-history.md, test-cases.md, change-summary.md, project docs, or current browser state clearly identify a URL, use it.
- If repository scripts, test files, package manifests, Makefiles, framework commands, or docs clearly identify a safe command or test filter for a CLI/backend/API/docs case, use it.
- If no URL, command, startup state, credentials location, test filter, or required fixture is clear, ask the user for the missing detail, then persist only the non-sensitive reusable facts to
agent-context.md.
- Do not invent credentials, URLs, commands, fixtures, or environment assumptions.
- Use the target environment and authorizations collected in Step 1.1. Reconfirm if navigation redirects to a different host or a more sensitive environment.
Browser Replay Contract
For every browser-ui case, and every hybrid case with browser steps:
- Use
<browser_replay_dir>/TC-NNN.js as the canonical replay artifact. Keep one case per file so a failed case can be rerun independently.
- Write the smallest Playwright-compatible
async (page) => { ... } expression. This is the format accepted by browser run-code capabilities. Do not add imports, a test framework, fixtures, generated wrappers, or project dependencies. Include navigation, stable selectors, inputs/actions, and explicit assertions that throw on mismatch.
- Start every canonical script with exactly these metadata lines, followed by the async expression:
// aif-case-digest: <case_digest>
// aif-target-fingerprint: <target_fingerprint>
Use the quoted placeholder "AIF_BASE_URL" for navigation instead of persisting an absolute target URL. Immediately before execution, replace that quoted placeholder in memory with a correctly JSON-encoded normalized_base_url; do not modify the saved file. Never write passwords, tokens, cookies, authorization values, one-time codes, personal secrets, or token-bearing URLs into the script. Reuse an already authenticated browser session or obtain secrets through the authorized runtime mechanism.
- A replay is
matching only when all three bindings agree: embedded case_digest, embedded target_fingerprint, and actual script_digest versus the last proof recorded in qa-check.md. Verify all three before every replay. A target mismatch MUST stop execution until the currently displayed and authorized target is verified and the script is explicitly rebound; rebinding changes the script digest and requires a new proof run.
- Create a missing or outdated script before its first execution whenever the case is mechanically clear. Compute its
script_digest, execute it once, and use that first execution as its proof run; do not automatically execute a new or updated script a second time.
- If interactive browser work was required to clarify the mechanics and the saved script changed after the observed run, mark the new script
Unproven. Do not immediately rerun it.
- An unproven or changed script may run only after its stated preconditions have been explicitly restored or verified. If the case can create, submit, send, delete, mutate permissions/settings, or cause another external side effect, obtain a new per-case authorization that explicitly says this is a repeat execution. Cleanup/reset actions remain subject to the same safety rules and authorization.
- Record
Script digest as a current evidence binding only after that exact file content has executed. For an unproven file, keep Script digest: n/a, set Proof status: Unproven, and record its computed candidate digest in the comment/evidence until a proof run completes.
- Before modifying any existing canonical script for repair, rebinding, or clarified mechanics, compute its old script digest and preserve it verbatim as
<browser_replay_dir>/history/TC-NNN-<old_script_digest>.js. Record the archived path, old digest, and initial error in qa-check.md. Never execute history files automatically.
- Classify replay failures without hiding product regressions:
- Syntax or browser-runner failure → automation defect; archive, repair, recompute
script_digest, and require a new proof run under the precondition/side-effect rules above.
- Missing selector → first preserve the original script and initial browser error/state. Repair the locator only when concrete browser evidence proves the same semantic element and expected behavior are still present and only locator mechanics changed. Record that proof. Otherwise mark the case
Failed; never choose a merely similar element or bypass the missing step.
- Script reaches an assertion and observed behavior differs → mark the case
Failed; this is regression/fix evidence, not a reason to regenerate the script.
- On any run after revision, worktree, or manual build changes, replay all current matching browser scripts, including cases that previously passed and failed. The capability/safety preflight and per-case side-effect authorization still apply to every replay. When safe and supported, execute non-stateful scripts in one browser session or batch to reduce setup and tool calls.
- Record the canonical path, case digest, script digest, target fingerprint, proof status, execution capability, and result in the case's
Browser replay and Evidence fields in qa-check.md. Proven means that exact script content executed; it does not mean the product result passed.
- If the available in-app Browser supports only discrete actions, read the saved script as the exact action/assertion program and replay it through those actions. If Playwright MCP can execute the file body directly, prefer direct execution. Do not regenerate known steps merely because the execution surface changed.
- Treat replay as the regression baseline, not the only browser check. After replay, decide whether one bounded exploratory pass is justified from concrete change risk:
- Run it when the change affects shared UI, layout, navigation, authentication, permissions, state transitions, or another cross-cutting flow; when the bug cause or fix behavior remains uncertain; when a matching replay needed repair because application behavior changed; or when replay exposed unexpected adjacent behavior.
- Skip it when the fix is narrow and isolated, matching replay directly exercises the changed behavior, no adjacent high-risk path is affected, and replay passes.
- When running it, inspect only directly affected adjacent routes, states, or edge inputs. Do not start open-ended browsing or regenerate working replay steps.
- Record
Browser exploration: Ran or Browser exploration: Skipped, the reason, scope, and concise redacted findings in qa-check.md.
- A new finding does not silently become a source test case. Record it in
qa-check.md and recommend adding it through /aif-qa test-cases; create its replay script after it has a TC-NNN case.
For each pending or selected case:
- Select the execution surface from the case classification:
browser-ui: execute the matching saved browser replay first; create or prove it only when missing or nonmatching by case, target, or script digest, then navigate with Browser or Playwright MCP and execute UI steps.
cli: run the relevant project command and inspect exit code/output.
backend-test: find and run the narrowest existing relevant test command or test filter first; broaden only when needed for confidence. Passing automated coverage is valid pass evidence for the case.
api: call the endpoint through existing project tooling, safe local commands, or browser/network capability as appropriate.
file-docs: inspect generated files, docs, config, or repository contents with Read/Grep/Glob and, when useful, project validation commands.
database-read: use only safe read-only checks unless the user explicitly authorizes mutation in a test-scoped environment.
hybrid: combine the necessary surfaces and record each evidence type.
- Execute the test steps as written, adapting only mechanics that are obvious from the selected surface.
- If execution stalls because of missing credentials, missing browser fixture, unavailable services, unclear setup, blocked navigation, missing selector knowledge, ambiguous field mapping, unknown route, unknown command, missing fixture, or unclear test filter, search
agent-context.md and agent-history.md first. For browser/UI auth or user-state gaps, try safe local/test fixture discovery or creation as described in Step 1.1 before blocking. If the answer or fixture is not already available, ask the user exactly for the missing information or authorization before marking the case blocked.
- When the user provides usable non-sensitive information, continue the case in the same run. Update
agent-context.md with durable cross-QA facts and append the resolved friction to agent-history.md only if it is likely to recur in future QA sets. Keep case-specific details in qa-check.md.
- If the user cannot provide the missing information, the service remains unavailable, authorization is denied, the required capability is unavailable, or a side-effect risk remains unresolved, keep the checkbox unchecked (
[ ]), set status to Blocked, and write the blocker.
- Compare the observed result to the expected result.
- If observed behavior matches, mark the case checked (
[x]), set status to Passed, and add concise redacted evidence. Evidence may be a browser observation, command and exit status, test name and result, API response summary, file path and matched condition, or other concrete observation.
- If observed behavior does not match, keep the checkbox unchecked (
[ ]), set status to Failed, and write a concrete redacted problem comment with observed vs expected behavior.
- Save
qa-check.md after every case.
Before the final summary, write detailed per-run evidence to qa-check.md, not to root-level memory. This includes the current branch/QA target, commands executed, exit codes, assertion counts, supporting file checks, case mappings, summary counts, and one-off blockers.
After writing qa-check.md, extract only reusable cross-QA learnings:
- Update
<paths.qa>/agent-context.md when a discovered fact is durable and likely useful for future QA sets.
- Append to
<paths.qa>/agent-history.md only when a blocker, selector, command pattern, test-filter convention, API endpoint pattern, route, setup note, or authorization decision is likely to recur beyond the current QA set.
- It is valid to append nothing to
agent-history.md for a successful run that produced no new reusable learning.
- Never append a full run transcript, branch/QA target, per-run summary counts, exact assertion totals, or current QA-plan label to
agent-history.md.
Keep <paths.qa>/agent-context.md concise and current; prefer stable facts over one-off observations. Do not write raw passwords, tokens, cookies, authorization headers, one-time codes, private personal data, token-bearing URLs, branch names, QA target paths, or run summaries.
Use screenshots or browser state observations when the active runtime makes them available, but do not require screenshots to pass a test.
Step 4: Final Summary
After stopping or finishing, report in ui_language:
- total test cases
- passed
- failed
- blocked
- pending
- path to
qa-check.md
- path to
browser-replay/ when browser scripts were created or executed
- path to
agent-context.md and agent-history.md when agent mode ran or requested missing execution context
- next recommended action
If mode = agent and one or more current cases are Blocked, split them into human-verifiable blocked cases and automation-only blocked cases before ending.
- Offer human-guided continuation only for blocked cases whose execution surface is
human, browser-ui, hybrid with human-observable steps, or unknown.
- Do NOT offer human-guided continuation for blocked
backend-test, cli, api, file-docs, or database-read cases when the blocker is missing automated coverage, missing command/test filter, unavailable service, or another technical automation prerequisite. For those cases, recommend adding/running the appropriate automated check or providing the missing command/fixture/context.
- If at least one human-verifiable blocked case exists, ask the user in
ui_language whether to run those eligible blocked cases now in human-guided mode.
- If the user agrees, continue in the same invocation using Step 2, but only for eligible blocked cases. Preserve existing agent blocker comments as context and append the human result/comment under the same case.
- If the user declines or stops, keep those cases unchecked and
Blocked.
- Write the handoff decision and any resolved blocked cases to
qa-check.md. Append to agent-history.md only when the handoff revealed a reusable cross-QA lesson.
If any case failed, the next recommended action should be to fix the issue and rerun /aif-qa-check <mode> <resolved_branch>.
If human-verifiable cases remain blocked after an agent-mode run and the user did not continue them in human mode, the next recommended action should be to run /aif-qa-check human <resolved_branch> for those cases or provide the missing execution context. If automation-only cases remain blocked, recommend adding/running the required automated test, command, fixture, service, or context instead. Do not imply that a blocked case is a product defect unless there is concrete observed evidence.
Artifact Ownership and Config Policy
- Primary ownership:
<paths.qa>/<branch-slug>/qa-check.md, canonical <paths.qa>/<branch-slug>/browser-replay/TC-NNN.js, preserved <paths.qa>/<branch-slug>/browser-replay/history/*.js, <paths.qa>/agent-context.md, and <paths.qa>/agent-history.md.
- Read policy: may read
<paths.qa>/<branch-slug>/change-summary.md, test-plan.md, test-cases.md, <paths.qa>/agent-context.md, and <paths.qa>/agent-history.md as QA context.
- Write policy: persistent writes are limited to
qa-check.md, branch-specific browser-replay/TC-NNN.js and browser-replay/history/*.js, agent-context.md, and agent-history.md; do not rewrite test-cases.md, test-plan.md, change-summary.md, or config.yaml.
- Config policy: config-aware, read-only. Reads
paths.description, paths.architecture, paths.qa, language.ui, language.artifacts, language.technical_terms, and git.enabled; never writes config.yaml.
Critical Rules
- MUST NOT run without
test-cases.md.
- MUST show only one case at a time in human mode.
- MUST ask the user whether the case works in human mode.
- MUST preserve failed-user comment wording except mandatory redaction of sensitive values before writing.
- MUST NOT mark a case passed in agent mode without concrete evidence from an appropriate execution surface for that case.
- MUST NOT block non-browser cases merely because live browser automation is unavailable.
- MUST classify deterministic internal/backend/CLI/API/database invariant cases as automated checks and MUST NOT route them to human manual QA as a substitute for missing automated coverage.
- MUST bind current results to (
tested_revision and worktree_digest) or manual_build_id, plus source_digest and case_digests.
- MUST mark stale results stale when the tested revision, worktree digest, manual build id, full source digest, or per-case digest changes.
- MUST require explicit authorization for production/unknown targets and destructive or external-side-effect cases.
- MUST redact credentials, cookies, authorization values, tokens, and sensitive URL parameters before writing comments or evidence.
- MUST save progress after every case.
- MUST re
…(truncated)
1---2name: aif-qa-check3description: Executes QA test cases created by /aif-qa in human-guided or automated-agent mode, including reusable browser replay scripts. Use when you need to walk through QA one case at a time, record pass/fail results, or have an agent verify and rerun cases through browser, CLI, API, automated tests, or file/document checks.4---56# QA Check — Execute Test Cases78Runs the `test-cases.md` artifact produced by `/aif-qa` and records execution status in `qa-check.md`.9In agent mode, it also maintains reusable cross-QA execution memory under the QA root:10- `agent-context.md` — curated, current, reusable non-sensitive setup facts for future automated QA runs11- `agent-history.md` — append-only reusable learnings extracted from prior runs, not a per-run audit log1213For browser/UI cases, agent mode saves executable replay scripts under the branch-specific `browser-replay/` directory. Later runs execute those scripts first, including scripts for previously passed cases, so fixes are retested and prior behavior gets regression coverage without regenerating browser automation.1415The skill is stack-free and agent-free: it does not assume a framework, package manager, browser tool name, or agent runtime. In agent mode, it uses the appropriate available execution surface for each case: live browser automation, CLI commands, project test runners, HTTP/API calls, database-safe read checks, file/document inspection, or other non-destructive automated verification.1617## Modes1819| Argument | Mode | What you do |20|----------|------|-------------|21| `human` | Human-guided QA | Show exactly one test case, ask the user whether it works, and record their answer |22| `agent` | Automated-agent QA | Execute test cases through the most appropriate available capability and record observed results |2324If no mode is provided, ask the user which mode to run.2526---2728## Workflow2930### Step 0: Load Config3132**FIRST:** Read `.ai-factory/config.yaml` if it exists to resolve:33- **Paths:** `paths.description`, `paths.architecture`, `paths.qa` (default: `.ai-factory/qa`)34- **Language:**35 - `language.ui` for AskUserQuestion prompts, progress messages, summaries, and next-step guidance36 - `language.artifacts` for the persisted `qa-check.md` artifact37 - `language.technical_terms` for human-readable technical terminology style in the artifact38 - If `language.artifacts` is missing, use `language.ui`39 - If both are missing, use `en`40- **Git:** `git.enabled` for branch resolution4142If config.yaml does not exist, use defaults:43- DESCRIPTION.md: `.ai-factory/DESCRIPTION.md`44- ARCHITECTURE.md: `.ai-factory/ARCHITECTURE.md`45- QA artifacts: `.ai-factory/qa/`46- `ui_language`: `en`47- `artifact_language`: `en`48- `technical_terms_policy`: `keep`49- Git enabled: `true`5051Store:52- `ui_language = language.ui || "en"`53- `artifact_language = language.artifacts || language.ui || "en"`54- `technical_terms_policy = language.technical_terms || "keep"`55- `git_enabled = git.enabled` when present, otherwise `true`56- `qa_root = resolved paths.qa || ".ai-factory/qa/"`57- `qa_agent_context_path = <qa_root>/agent-context.md`58- `qa_agent_history_path = <qa_root>/agent-history.md`5960All AskUserQuestion prompts, user-visible explanations, per-case summaries, and final summaries MUST be written in `ui_language`.6162The persisted `qa-check.md` artifact MUST be written in `artifact_language`.6364Templates define structure, not language. Use the canonical English template in `templates/QA-CHECK.md`. If `artifact_language` is not `en`, translate headings, labels, status text, comments you author, and explanatory text to `artifact_language` before saving. Preserve checkbox syntax, test case IDs (`TC-001`), commands, paths, config keys, URLs, selectors, package names, API names, branch names, and raw error messages.6566For `artifact_language = ru`, write human-readable prose, headings, statuses, summaries, and agent-authored comments in Russian. Preserve user wording except mandatory redaction of sensitive values before writing.6768Apply `technical_terms_policy` while writing artifacts:69- `keep` — keep common technical terms such as `browser`, `selector`, `viewport`, `endpoint`, `payload`, `regression`, and `fixture` when clearer70- `translate` — translate human-readable technical terms where a natural target-language term exists71- `mixed` — translate ordinary prose terms while keeping code, infrastructure, and ecosystem terms unchanged7273### Step 0.1: Load Project Context7475Read the resolved description path if it exists.7677Read the resolved architecture path if it exists.7879Read `.ai-factory/skill-context/aif-qa-check/SKILL.md` — MANDATORY if the file exists. Treat it as project-level overrides for this skill.8081Read `<paths.qa>/agent-context.md` if it exists. Treat it as reusable, user-approved QA execution context for future automated runs: stable environment classes, canonical local/staging URLs, login route, test account role, allowed non-sensitive usernames, required seed data patterns, stable selectors, field identifiers, known redirects, startup prerequisites, safe command patterns, reusable test-filter conventions, environment notes, and service startup instructions.8283Read `<paths.qa>/agent-history.md` if it exists. Treat it as append-only cross-QA memory from prior `/aif-qa-check` runs. Before asking the user or attempting automated execution, search it for reusable prior answers to the same blocker so the skill does not repeat avoidable mistakes.8485`agent-context.md` and `agent-history.md` are global to all QA sets under `paths.qa`. They MUST NOT become logs for a specific QA plan. Do not write branch names, branch slugs, QA target paths, current `artifact_dir`, `TC-*` mappings, per-run summary counts, exhaustive command transcripts, assertion counts from a single run, one-off file checks, or plan-specific labels such as a numbered QA folder name. Write those details only to `<paths.qa>/<branch-slug>/qa-check.md`.8687Only promote information into `agent-context.md` or `agent-history.md` when it is likely to help future unrelated QA runs. Prefer stable patterns over concrete one-off run facts. Example: `Laravel backend checks use php artisan test --filter=<TestClass>` is reusable; `05-profile-formula-recommendations ran ProfileReaderTest with 4 tests / 13 assertions` is run-specific and belongs only in `qa-check.md`.8889Never write global negative capability or scope claims unless they are true for the whole project. Do not write statements like `browser automation is not required`, `browser URL is not used`, `all QA is backend/CLI`, or `these cases are backend-test/cli/file-docs` to `agent-context.md` or `agent-history.md` when that conclusion came from one QA plan. Instead, keep it branch-specific in `qa-check.md`, or phrase a reusable conditional rule such as `For backend-only QA cases, browser automation is not required; browser/UI cases still require Browser or Playwright MCP`.9091Never treat either file as a source of secrets. If either file contains an apparent password, token, cookie, authorization header, one-time code, or other secret, ignore that value for execution and redact it the next time the file is rewritten or appended to.9293### Step 0.2: Parse Arguments and Resolve Branch9495Parse `$ARGUMENTS`:96971. Detect mode: first word matching `human` or `agent`; remove it from arguments.982. Detect branch: remaining text, if any, is the branch label.993. If no mode was provided, ask in `ui_language` which mode to run:100 - Human-guided QA101 - Automated-agent QA102103Resolve the working branch:104105```text106If git_enabled = false or the repository is not a git work tree:107 If branch was provided in arguments → use it as the resolved branch label108 Otherwise → set resolved_branch = "manual"109If git_enabled = true and the repository is a git work tree:110 If branch was provided in arguments → use it as the resolved branch111 Otherwise → run: git branch --show-current112```113114Store:115- `resolved_branch`116- `artifact_dir = <resolved paths.qa>/<branch-slug>`117- `test_cases_path = <artifact_dir>/test-cases.md`118- `qa_check_path = <artifact_dir>/qa-check.md`119- `browser_replay_dir = <artifact_dir>/browser-replay`120121Compute `branch-slug` with the exact same algorithm as `/aif-qa`:1221. Replace every character not in `[A-Za-z0-9._-]` with `-`, collapse consecutive `-`, trim leading/trailing `-`, and use `branch` if empty. Then MUST truncate `safe_slug` to the first 40 ASCII characters. Because the normalized `safe_slug` alphabet is `[A-Za-z0-9._-]`, byte length and character length are identical.1232. Run `git hash-object --stdin <<< "<resolved_branch>"` and take the first 8 hex characters as `hash8`.1243. Combine: `branch-slug = "<safe_slug>-<hash8>"`.125126### Step 1: Verify QA Inputs127128Check for `<artifact_dir>/test-cases.md`.129130If it is missing, STOP and ask in `ui_language` whether to:1311. Run `/aif-qa test-cases <resolved_branch>` first1322. Cancel133134Read `test-cases.md`. Extract test cases by IDs (`TC-001`, `TC-002`, etc.), titles, priority, type, execution surface when present, preconditions, steps, expected results, and test data. Preserve their order.135136Compute source binding metadata before creating or modifying `qa-check.md`:137- `source_digest` — deterministic digest of the full `test-cases.md` content using `git hash-object --no-filters <test_cases_path>` when available, or `git hash-object --stdin` over the exact file content.138- `case_digests` — deterministic per-case digest for each extracted `TC-NNN`, computed from that case's canonical block including title, priority, type, preconditions, steps, expected result, and test data.139- `tested_revision` — when `git_enabled = true` and the repository is a git work tree, run `git rev-parse HEAD` and record the resolved commit SHA.140- `worktree_digest` — when `git_enabled = true` and the repository is a git work tree, record a deterministic digest of the current working tree state so dirty-tree QA cannot be reused after local changes without a commit.141- `manual_build_id` — when `git_enabled = false` or the repository is not a git work tree, ask the user for an explicit build/version identifier before creating or resuming results. Do not accept an empty identifier.142- `replay_script_digests` — for each existing canonical `<browser_replay_dir>/TC-NNN.js`, hash the exact file content with `git hash-object --no-filters <path>` when available, or another stable SHA-1/SHA-256 digest. This binds evidence to the automation that actually ran even though QA-owned replay files are excluded from `worktree_digest`.143144Canonicalize each per-case digest input exactly:1451. Extract the raw Markdown block from the line containing the case's `TC-NNN` identifier through the line before the next `TC-NNN` block or end of file.1462. Normalize line endings from CRLF or CR to LF.1473. Remove trailing spaces and tabs from each line.1484. Trim leading and trailing blank lines from the extracted block.1495. Preserve the original field order, bullet markers, indentation, internal blank lines, Markdown punctuation, and all remaining whitespace. Do not sort fields, collapse whitespace, or rewrite bullets.1506. Wrap the normalized block with raw block boundaries before hashing: `BEGIN TC-NNN\n<normalized block>\nEND TC-NNN\n`.1517. Hash the wrapped block with `git hash-object --stdin` when available, or another stable SHA-1/SHA-256 digest if git is unavailable.152153Compute `worktree_digest` exactly when git is enabled and a git work tree exists:1541. Capture `git status --porcelain=v1 --untracked-files=all`.1552. Capture `git diff --binary HEAD --`.1563. Exclude `qa_check_path` and every file under `browser_replay_dir` from the status, diff, and untracked-file digest inputs so QA-owned result/replay artifacts do not stale their own results. Do not exclude `test_cases_path`; source changes are also tracked by `source_digest`.1574. For each remaining untracked file listed by porcelain status, append `UNTRACKED <path> <content-digest>` where `<content-digest>` is `git hash-object --no-filters <path>` when the file is readable.1585. Normalize line endings in the combined input to LF and hash it with `git hash-object --stdin`.1596. If the filtered work tree input is clean, record the digest of the canonical string `clean\n`.160161If `mode = agent`, perform Step 1.1 before creating or modifying `qa-check.md`. Existing `qa-check.md` may be inspected read-only during this gate.162163If `qa-check.md` exists, read it and resume from existing statuses only after comparing stored binding metadata to the current binding metadata:164- If `tested_revision` changed, mark every prior result as `Stale` and unchecked, preserve prior comments/evidence as historical context, and require retest. Do not count stale pass/fail/block statuses as current.165- If `worktree_digest` changed, mark every prior result as `Stale` and unchecked, preserve prior comments/evidence as historical context, and require retest. Do not count stale pass/fail/block statuses as current.166- If `manual_build_id` changed, treat it the same as a tested revision change.167- If `source_digest` changed, compare `case_digests`. Preserve current status only for cases whose per-case digest is unchanged and whose tested revision/manual build id and worktree digest are unchanged.168- If an existing case's digest changed, mark that case `Stale`, unchecked, and require retest.169- If a new case appears, add it as unchecked `Pending`.170- If a prior case no longer exists in `test-cases.md`, keep its historical entry marked `Stale` or move it to an artifact-language "Stale / Removed Cases" section; never count it as current.171- Do not overwrite prior pass/fail comments unless the user explicitly chooses to retest that case.172- If a canonical replay file's actual content digest differs from the `Script digest` recorded for that case, mark the case `Stale`, preserve the previous replay metadata/evidence, and require a proof run of the changed script. Missing legacy script-digest or target-fingerprint metadata means the script is unproven, not matching.173- Revision, worktree, or manual-build staleness alone does not invalidate a browser replay script. It remains reusable only while its embedded case digest, embedded target fingerprint, and actual script digest match the current case, approved target, and previously recorded script digest.174175If `qa-check.md` does not exist, create it from `templates/QA-CHECK.md` using the extracted test cases and source binding metadata. Every case starts unchecked and `Pending`.176177### Step 1.1: Agent Mode Capability and Safety Preflight178179Agent mode is user-only (`disable-model-invocation: true`) because it can perform live browser actions, shell commands, local service calls, and other checks with meaningful side effects.180181Before executing any case or writing `qa-check.md` in agent mode:1821. Detect available execution capabilities from the current runtime tools:183 - shell/CLI command execution184 - project test runners and build tools available through shell185 - live browser automation through in-app Browser or Playwright MCP186 - HTTP/API checks available through existing project tooling or safe shell commands187 - file, document, and repository inspection through Read/Grep/Glob1882. Classify each test case by the evidence surface it requires: `browser-ui`, `cli`, `backend-test`, `api`, `file-docs`, `database-read`, `hybrid`, `human`, or `unknown`. If the case has an explicit `Execution surface:` field, use it unless the case text clearly contradicts it; record any override in `agent-history.md`.1893. Choose the least invasive capability that can produce concrete evidence for that case. Browser automation is REQUIRED for browser/UI-observable cases and hybrid cases whose expected result depends on rendered web behavior. Do not substitute CLI commands, unit tests, static inspection, or backend checks for a browser/UI case; those may be recorded only as supporting evidence.1904. If a case requires browser automation, prefer the in-app Browser capability if available. If in-app Browser is not available, use Playwright MCP if available.191 - Prefer a browser capability that can execute a saved Playwright-compatible `page` script when replay scripts exist.1925. If a browser/UI case requires live browser automation and neither in-app Browser nor Playwright MCP is available, keep only that case unchecked, set status to `Blocked`, ask the user in `ui_language` to enable a live browser capability, and append this blocker to `<paths.qa>/agent-history.md`. Continue with other cases that can be verified through CLI, tests, API, or file/document checks.1936. Use `<paths.qa>/agent-context.md`, `<paths.qa>/agent-history.md`, `test-cases.md`, `change-summary.md`, project docs, current browser state, and repository scripts to determine the target URL, service startup commands, test commands, and environment.1947. For browser/UI cases that require authentication or specific user state, resolve a browser test identity before marking the case blocked:195 - First search `agent-context.md`, `agent-history.md`, test data, seeders/factories/fixtures, docs, local database-safe reads, and existing test users for a matching non-sensitive test identity.196 - If the environment is clearly `local`, `test`, or disposable `development`, and repository tooling provides a safe way to create or seed a synthetic test user/state, create the minimal disposable fixture needed for the case. Record the fixture identifier and setup command/evidence in `qa-check.md`.197 - If creating a fixture requires a password, role, tenant, workspace, feature flag, seed command, or any unclear setup decision, ask the user before blocking. Ask whether to provide an existing test account or authorize creating a disposable fixture.198 - If the user provides or authorizes disposable test credentials for a `local` or `test` environment, use them for this run. Store credentials in `agent-context.md` only if the user explicitly says they are reusable test-only credentials and safe to persist. Otherwise store only the username/role and where to retrieve the secret.199 - Never store production credentials, personal credentials, cookies, session tokens, one-time codes, or shared secrets in `agent-context.md` or `agent-history.md`.2008. If a user-suppliable prerequisite is missing, ask the user before blocking:201 - application URL or login URL202 - test account role or non-sensitive username203 - where to find credentials without exposing them in artifacts204 - authorization to create a disposable local/test browser fixture and the desired initial state205 - required startup state, seed data, feature flag, tenant, organization, or workspace206 - ambiguous login mechanism or authentication provider207 - stable selector, field identifier, or page route needed to continue208 - exact project command, test filter, service startup command, or safe API endpoint needed to continue2099. Save reusable non-sensitive answers to `<paths.qa>/agent-context.md` immediately only when the answer is general enough to help future QA sets. Append the redacted answer summary to `<paths.qa>/agent-history.md` only when it captures a reusable lesson; otherwise keep it in the branch-specific `qa-check.md` evidence/comment.21010. Determine and display the target environment, including URL when applicable and inferred environment class: `local`, `development`, `staging`, `test`, `production`, or `unknown`. For a browser target:211 - Derive `normalized_base_url` from the resolved target URL by removing user information, query, and fragment; preserving scheme, host, port, and deployment base path; and removing a trailing slash except for the origin root.212 - Compute `target_fingerprint` from the exact canonical input `AIF TARGET\n<environment_class>\n<normalized_base_url>\n` with `git hash-object --stdin` when available, or another stable SHA-1/SHA-256 digest.213 - Record the environment class and `target_fingerprint` in `qa-check.md`. Never treat a replay created for another target fingerprint as matching.21411. Do not execute browser actions, API writes, CLI commands with side effects, database writes, destructive commands, payment, permission, email, notification, data export, or other external-side-effect steps against `production` or `unknown` targets without explicit user authorization immediately before execution. Record the authorization decision and target class in `qa-check.md`. Append it to `agent-history.md` only when it creates a reusable cross-QA policy or recurring authorization lesson; do not persist broad production authorization for future runs.21512. Inspect each case and proposed command for destructive, irreversible, payment, permission, email, notification, data export, database mutation, service restart, deployment, filesystem deletion, or other external-side-effect risk. Require explicit per-case authorization before executing any such case.21613. Prefer read-only and test-scoped commands. Do not run destructive shell commands such as `rm`, `git reset`, `git checkout --`, database resets, migrations against shared environments, deploy commands, or cleanup commands that delete user data unless the user explicitly authorizes the exact action.21714. If authorization is denied or unclear, leave the case unchecked, set status to `Blocked`, and write the blocker to `qa-check.md` without executing the risky action. Append the decision to `agent-history.md` only when it is a reusable cross-QA lesson, not merely a case-specific denial.218219Redaction is mandatory for agent comments/evidence and all human-entered comments/evidence:220- Redact credentials, cookies, authorization values, session tokens, API keys, bearer tokens, basic auth values, one-time codes, and personal secrets.221- Redact sensitive URL parameters such as `token`, `access_token`, `refresh_token`, `id_token`, `code`, `secret`, `password`, `passwd`, `pwd`, `auth`, `key`, `api_key`, `session`, `sid`, and `jwt`.222- Replace redacted values with `[REDACTED]` before writing comments or evidence to `qa-check.md`.223- Do not paste raw browser storage, cookies, request headers, authorization headers, or token-bearing URLs into the artifact.224225### Step 2: Human Mode226227Run exactly one pending or selected test case at a time.228229For each case:2301. Show only that test case's title, priority, preconditions, steps, test data, and expected result.2312. Include a short per-case summary in `ui_language`.2323. The summary question MUST mean: "Test this and answer whether it works." For `ui_language = ru`, use exactly: `Протестируйте и ответьте работает или нет`.2334. Ask the user for the result:234 - Works / completed235 - Does not work / failed236 - Stop for now2375. If the user says it works, mark the case as checked (`[x]`) in `qa-check.md`, set status to `Passed`, and add the current mode as `human`.2386. If the user says it failed, keep the checkbox unchecked (`[ ]`), set status to `Failed`, ask for the reason, and write the user's explanation as the comment while preserving user wording except mandatory redaction of sensitive values.2397. Save `qa-check.md` after every case so progress survives context resets.240241Do not show the next test case until the current one has a result or the user stops.242243### Step 3: Agent Mode244245Agent mode MUST produce concrete, case-appropriate evidence. Browser execution, CLI output, automated test output, API responses, backend command results, database-safe read results, or file/document inspection can each be enough to mark a case passed when that evidence directly covers the case steps and expected result.246247Internal deterministic invariants MUST be treated as automated checks, not manual QA. Cases involving service-method calls, cache state, materialized rows, exact arrays, raw database values, formula outputs, CLI internals, or similar non-human-observable behavior should be classified as `backend-test`, `cli`, `api`, or `database-read` as appropriate. First look for existing tests or commands that already cover the invariant. If existing tests are found and pass, mark the corresponding `TC-*` case `Passed` with the test class/name, command/filter, exit status, and assertion/result summary as evidence. If one test run covers multiple `TC-*` cases, reuse that evidence on every covered case and optionally summarize the shared run under "Supporting Automated Checks". If no suitable automated coverage exists, keep the case `Blocked` with a blocker such as `Missing automated test coverage for backend invariant`; do not ask the user to manually verify internal arrays or raw database values.248249Determine the test target:250- If `agent-context.md`, `agent-history.md`, `test-cases.md`, `change-summary.md`, project docs, or current browser state clearly identify a URL, use it.251- If repository scripts, test files, package manifests, Makefiles, framework commands, or docs clearly identify a safe command or test filter for a CLI/backend/API/docs case, use it.252- If no URL, command, startup state, credentials location, test filter, or required fixture is clear, ask the user for the missing detail, then persist only the non-sensitive reusable facts to `agent-context.md`.253- Do not invent credentials, URLs, commands, fixtures, or environment assumptions.254- Use the target environment and authorizations collected in Step 1.1. Reconfirm if navigation redirects to a different host or a more sensitive environment.255256#### Browser Replay Contract257258For every `browser-ui` case, and every `hybrid` case with browser steps:2592601. Use `<browser_replay_dir>/TC-NNN.js` as the canonical replay artifact. Keep one case per file so a failed case can be rerun independently.2612. Write the smallest Playwright-compatible `async (page) => { ... }` expression. This is the format accepted by browser run-code capabilities. Do not add imports, a test framework, fixtures, generated wrappers, or project dependencies. Include navigation, stable selectors, inputs/actions, and explicit assertions that throw on mismatch.2623. Start every canonical script with exactly these metadata lines, followed by the async expression:263 - `// aif-case-digest: <case_digest>`264 - `// aif-target-fingerprint: <target_fingerprint>`265 Use the quoted placeholder `"AIF_BASE_URL"` for navigation instead of persisting an absolute target URL. Immediately before execution, replace that quoted placeholder in memory with a correctly JSON-encoded `normalized_base_url`; do not modify the saved file. Never write passwords, tokens, cookies, authorization values, one-time codes, personal secrets, or token-bearing URLs into the script. Reuse an already authenticated browser session or obtain secrets through the authorized runtime mechanism.2664. A replay is `matching` only when all three bindings agree: embedded `case_digest`, embedded `target_fingerprint`, and actual `script_digest` versus the last proof recorded in `qa-check.md`. Verify all three before every replay. A target mismatch MUST stop execution until the currently displayed and authorized target is verified and the script is explicitly rebound; rebinding changes the script digest and requires a new proof run.2675. Create a missing or outdated script before its first execution whenever the case is mechanically clear. Compute its `script_digest`, execute it once, and use that first execution as its proof run; do not automatically execute a new or updated script a second time.268 - If interactive browser work was required to clarify the mechanics and the saved script changed after the observed run, mark the new script `Unproven`. Do not immediately rerun it.269 - An unproven or changed script may run only after its stated preconditions have been explicitly restored or verified. If the case can create, submit, send, delete, mutate permissions/settings, or cause another external side effect, obtain a new per-case authorization that explicitly says this is a repeat execution. Cleanup/reset actions remain subject to the same safety rules and authorization.270 - Record `Script digest` as a current evidence binding only after that exact file content has executed. For an unproven file, keep `Script digest: n/a`, set `Proof status: Unproven`, and record its computed candidate digest in the comment/evidence until a proof run completes.2716. Before modifying any existing canonical script for repair, rebinding, or clarified mechanics, compute its old script digest and preserve it verbatim as `<browser_replay_dir>/history/TC-NNN-<old_script_digest>.js`. Record the archived path, old digest, and initial error in `qa-check.md`. Never execute history files automatically.2727. Classify replay failures without hiding product regressions:273 - Syntax or browser-runner failure → automation defect; archive, repair, recompute `script_digest`, and require a new proof run under the precondition/side-effect rules above.274 - Missing selector → first preserve the original script and initial browser error/state. Repair the locator only when concrete browser evidence proves the same semantic element and expected behavior are still present and only locator mechanics changed. Record that proof. Otherwise mark the case `Failed`; never choose a merely similar element or bypass the missing step.275 - Script reaches an assertion and observed behavior differs → mark the case `Failed`; this is regression/fix evidence, not a reason to regenerate the script.2768. On any run after revision, worktree, or manual build changes, replay all current matching browser scripts, including cases that previously passed and failed. The capability/safety preflight and per-case side-effect authorization still apply to every replay. When safe and supported, execute non-stateful scripts in one browser session or batch to reduce setup and tool calls.2779. Record the canonical path, case digest, script digest, target fingerprint, proof status, execution capability, and result in the case's `Browser replay` and `Evidence` fields in `qa-check.md`. `Proven` means that exact script content executed; it does not mean the product result passed.27810. If the available in-app Browser supports only discrete actions, read the saved script as the exact action/assertion program and replay it through those actions. If Playwright MCP can execute the file body directly, prefer direct execution. Do not regenerate known steps merely because the execution surface changed.27911. Treat replay as the regression baseline, not the only browser check. After replay, decide whether one bounded exploratory pass is justified from concrete change risk:280 - Run it when the change affects shared UI, layout, navigation, authentication, permissions, state transitions, or another cross-cutting flow; when the bug cause or fix behavior remains uncertain; when a matching replay needed repair because application behavior changed; or when replay exposed unexpected adjacent behavior.281 - Skip it when the fix is narrow and isolated, matching replay directly exercises the changed behavior, no adjacent high-risk path is affected, and replay passes.282 - When running it, inspect only directly affected adjacent routes, states, or edge inputs. Do not start open-ended browsing or regenerate working replay steps.283 - Record `Browser exploration: Ran` or `Browser exploration: Skipped`, the reason, scope, and concise redacted findings in `qa-check.md`.284 - A new finding does not silently become a source test case. Record it in `qa-check.md` and recommend adding it through `/aif-qa test-cases`; create its replay script after it has a `TC-NNN` case.285286For each pending or selected case:2871. Select the execution surface from the case classification:288 - `browser-ui`: execute the matching saved browser replay first; create or prove it only when missing or nonmatching by case, target, or script digest, then navigate with Browser or Playwright MCP and execute UI steps.289 - `cli`: run the relevant project command and inspect exit code/output.290 - `backend-test`: find and run the narrowest existing relevant test command or test filter first; broaden only when needed for confidence. Passing automated coverage is valid pass evidence for the case.291 - `api`: call the endpoint through existing project tooling, safe local commands, or browser/network capability as appropriate.292 - `file-docs`: inspect generated files, docs, config, or repository contents with Read/Grep/Glob and, when useful, project validation commands.293 - `database-read`: use only safe read-only checks unless the user explicitly authorizes mutation in a test-scoped environment.294 - `hybrid`: combine the necessary surfaces and record each evidence type.2952. Execute the test steps as written, adapting only mechanics that are obvious from the selected surface.2963. If execution stalls because of missing credentials, missing browser fixture, unavailable services, unclear setup, blocked navigation, missing selector knowledge, ambiguous field mapping, unknown route, unknown command, missing fixture, or unclear test filter, search `agent-context.md` and `agent-history.md` first. For browser/UI auth or user-state gaps, try safe local/test fixture discovery or creation as described in Step 1.1 before blocking. If the answer or fixture is not already available, ask the user exactly for the missing information or authorization before marking the case blocked.2974. When the user provides usable non-sensitive information, continue the case in the same run. Update `agent-context.md` with durable cross-QA facts and append the resolved friction to `agent-history.md` only if it is likely to recur in future QA sets. Keep case-specific details in `qa-check.md`.2985. If the user cannot provide the missing information, the service remains unavailable, authorization is denied, the required capability is unavailable, or a side-effect risk remains unresolved, keep the checkbox unchecked (`[ ]`), set status to `Blocked`, and write the blocker.2996. Compare the observed result to the expected result.3007. If observed behavior matches, mark the case checked (`[x]`), set status to `Passed`, and add concise redacted evidence. Evidence may be a browser observation, command and exit status, test name and result, API response summary, file path and matched condition, or other concrete observation.3018. If observed behavior does not match, keep the checkbox unchecked (`[ ]`), set status to `Failed`, and write a concrete redacted problem comment with observed vs expected behavior.3029. Save `qa-check.md` after every case.303304Before the final summary, write detailed per-run evidence to `qa-check.md`, not to root-level memory. This includes the current branch/QA target, commands executed, exit codes, assertion counts, supporting file checks, case mappings, summary counts, and one-off blockers.305306After writing `qa-check.md`, extract only reusable cross-QA learnings:307- Update `<paths.qa>/agent-context.md` when a discovered fact is durable and likely useful for future QA sets.308- Append to `<paths.qa>/agent-history.md` only when a blocker, selector, command pattern, test-filter convention, API endpoint pattern, route, setup note, or authorization decision is likely to recur beyond the current QA set.309- It is valid to append nothing to `agent-history.md` for a successful run that produced no new reusable learning.310- Never append a full run transcript, branch/QA target, per-run summary counts, exact assertion totals, or current QA-plan label to `agent-history.md`.311312Keep `<paths.qa>/agent-context.md` concise and current; prefer stable facts over one-off observations. Do not write raw passwords, tokens, cookies, authorization headers, one-time codes, private personal data, token-bearing URLs, branch names, QA target paths, or run summaries.313314Use screenshots or browser state observations when the active runtime makes them available, but do not require screenshots to pass a test.315316### Step 4: Final Summary317318After stopping or finishing, report in `ui_language`:319- total test cases320- passed321- failed322- blocked323- pending324- path to `qa-check.md`325- path to `browser-replay/` when browser scripts were created or executed326- path to `agent-context.md` and `agent-history.md` when agent mode ran or requested missing execution context327- next recommended action328329If `mode = agent` and one or more current cases are `Blocked`, split them into human-verifiable blocked cases and automation-only blocked cases before ending.330- Offer human-guided continuation only for blocked cases whose execution surface is `human`, `browser-ui`, `hybrid` with human-observable steps, or `unknown`.331- Do NOT offer human-guided continuation for blocked `backend-test`, `cli`, `api`, `file-docs`, or `database-read` cases when the blocker is missing automated coverage, missing command/test filter, unavailable service, or another technical automation prerequisite. For those cases, recommend adding/running the appropriate automated check or providing the missing command/fixture/context.332- If at least one human-verifiable blocked case exists, ask the user in `ui_language` whether to run those eligible blocked cases now in human-guided mode.333- If the user agrees, continue in the same invocation using Step 2, but only for eligible blocked cases. Preserve existing agent blocker comments as context and append the human result/comment under the same case.334- If the user declines or stops, keep those cases unchecked and `Blocked`.335- Write the handoff decision and any resolved blocked cases to `qa-check.md`. Append to `agent-history.md` only when the handoff revealed a reusable cross-QA lesson.336337If any case failed, the next recommended action should be to fix the issue and rerun `/aif-qa-check <mode> <resolved_branch>`.338339If human-verifiable cases remain blocked after an agent-mode run and the user did not continue them in human mode, the next recommended action should be to run `/aif-qa-check human <resolved_branch>` for those cases or provide the missing execution context. If automation-only cases remain blocked, recommend adding/running the required automated test, command, fixture, service, or context instead. Do not imply that a blocked case is a product defect unless there is concrete observed evidence.340341## Artifact Ownership and Config Policy342343- Primary ownership: `<paths.qa>/<branch-slug>/qa-check.md`, canonical `<paths.qa>/<branch-slug>/browser-replay/TC-NNN.js`, preserved `<paths.qa>/<branch-slug>/browser-replay/history/*.js`, `<paths.qa>/agent-context.md`, and `<paths.qa>/agent-history.md`.344- Read policy: may read `<paths.qa>/<branch-slug>/change-summary.md`, `test-plan.md`, `test-cases.md`, `<paths.qa>/agent-context.md`, and `<paths.qa>/agent-history.md` as QA context.345- Write policy: persistent writes are limited to `qa-check.md`, branch-specific `browser-replay/TC-NNN.js` and `browser-replay/history/*.js`, `agent-context.md`, and `agent-history.md`; do not rewrite `test-cases.md`, `test-plan.md`, `change-summary.md`, or `config.yaml`.346- Config policy: config-aware, read-only. Reads `paths.description`, `paths.architecture`, `paths.qa`, `language.ui`, `language.artifacts`, `language.technical_terms`, and `git.enabled`; never writes `config.yaml`.347348## Critical Rules3493501. MUST NOT run without `test-cases.md`.3512. MUST show only one case at a time in human mode.3523. MUST ask the user whether the case works in human mode.3534. MUST preserve failed-user comment wording except mandatory redaction of sensitive values before writing.3545. MUST NOT mark a case passed in agent mode without concrete evidence from an appropriate execution surface for that case.3556. MUST NOT block non-browser cases merely because live browser automation is unavailable.3567. MUST classify deterministic internal/backend/CLI/API/database invariant cases as automated checks and MUST NOT route them to human manual QA as a substitute for missing automated coverage.3578. MUST bind current results to (`tested_revision` and `worktree_digest`) or `manual_build_id`, plus `source_digest` and `case_digests`.3589. MUST mark stale results stale when the tested revision, worktree digest, manual build id, full source digest, or per-case digest changes.35910. MUST require explicit authorization for production/unknown targets and destructive or external-side-effect cases.36011. MUST redact credentials, cookies, authorization values, tokens, and sensitive URL parameters before writing comments or evidence.36112. MUST save progress after every case.36213. MUST re363364…(truncated)