Webapp Testing

Quick Summary

Goal: Test local web applications using Python Playwright scripts with server lifecycle management.

For full-site QA audits (accessibility, performance, security, SEO), use test-ui instead.

Workflow:

1. Decide Approach — Static HTML (read selectors directly) vs dynamic app (use server helper)
2. Start Server — Use scripts/with_server.py for automatic server lifecycle management
3. Reconnaissance — Navigate, wait for networkidle, screenshot/inspect DOM
4. Execute Actions — Write Playwright scripts using discovered selectors

Key Rules:

• Always wait for networkidle before inspecting DOM on dynamic apps
• Use bundled scripts as black boxes; run --help first, don't read source
• Always launch Chromium in headless mode and close browser when done

Be skeptical. Apply critical thinking, sequential thinking. Every claim needs traced proof, confidence percentages (Idea should be more than 80%).

Web Application Testing

To test local web applications, write native Python Playwright scripts.

Helper Scripts Available:

• scripts/with_server.py - Manages server lifecycle (supports multiple servers)

Always run scripts with --help first to see usage. DO NOT read the source until you try running the script first and find that a customized solution is abslutely necessary. These scripts can be very large and thus pollute your context window. They exist to be called directly as black-box scripts rather than ingested into your context window.

Decision Tree: Choosing Your Approach

User task → Is it static HTML?
    ├─ Yes → Read HTML file directly to identify selectors
    │         ├─ Success → Write Playwright script using selectors
    │         └─ Fails/Incomplete → Treat as dynamic (below)
    │
    └─ No (dynamic webapp) → Is the server already running?
        ├─ No → Run: python scripts/with_server.py --help
        │        Then use the helper + write simplified Playwright script
        │
        └─ Yes → Reconnaissance-then-action:
            1. Navigate and wait for networkidle
            2. Take screenshot or inspect DOM
            3. Identify selectors from rendered state
            4. Execute actions with discovered selectors

Example: Using with_server.py

To start a server, run --help first, then use the helper:

Single server:

python scripts/with_server.py --server "npm run dev" --port 5173 -- python your_automation.py

Multiple servers (e.g., backend + frontend):

python scripts/with_server.py \
  --server "cd backend && python server.py" --port 3000 \
  --server "cd frontend && npm run dev" --port 5173 \
  -- python your_automation.py

To create an automation script, include only Playwright logic (servers are managed automatically):

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True) # Always launch chromium in headless mode
    page = browser.new_page()
    page.goto('http://localhost:5173') # Server already running and ready
    page.wait_for_load_state('networkidle') # CRITICAL: Wait for JS to execute
    # ... your automation logic
    browser.close()

Reconnaissance-Then-Action Pattern

1. Inspect rendered DOM:

   page.screenshot(path='/tmp/inspect.png', full_page=True)
   content = page.content()
   page.locator('button').all()
   

2. Identify selectors from inspection results

3. Execute actions using discovered selectors

Common Pitfall

❌ Don't inspect the DOM before waiting for networkidle on dynamic apps ✅ Do wait for page.wait_for_load_state('networkidle') before inspection

Best Practices

• Use bundled scripts as black boxes - To accomplish a task, consider whether one of the scripts available in scripts/ can help. These scripts handle common, complex workflows reliably without cluttering the context window. Use --help to see usage, then invoke directly.
• Use sync_playwright() for synchronous scripts
• Always close the browser when done
• Use descriptive selectors: text=, role=, CSS selectors, or IDs
• Add appropriate waits: page.wait_for_selector() or page.wait_for_timeout()

Reference Files

• examples/ - Examples showing common patterns:
  • element_discovery.py - Discovering buttons, links, and inputs on a page
  • static_html_automation.py - Using file:// URLs for local HTML
  • console_logging.py - Capturing console logs during automation

[IMPORTANT] Use TaskCreate to break ALL work into small tasks BEFORE starting — including tasks for each file read. This prevents context loss from long files. For simple tasks, AI MUST ATTENTION ask user whether to skip.

Critical Thinking Mindset — Apply critical thinking, sequential thinking. Every claim needs traced proof, confidence >80% to act. Anti-hallucination: Never present guess as fact — cite sources for every claim, admit uncertainty freely, self-check output for errors, cross-reference independently, stay skeptical of own confidence — certainty without evidence root of all hallucination.

Evidence-Based Reasoning — Speculation is FORBIDDEN. Every claim needs proof.

1. Cite file:line, grep results, or framework docs for EVERY claim
2. Declare confidence: >80% act freely, 60-80% verify first, <60% DO NOT recommend
3. Cross-service validation required for architectural changes
4. "I don't have enough evidence" is valid and expected output

BLOCKED until: - [ ] Evidence file path (file:line) - [ ] Grep search performed - [ ] 3+ similar patterns found - [ ] Confidence level stated

Forbidden without proof: "obviously", "I think", "should be", "probably", "this is because" If incomplete → output: "Insufficient evidence. Verified: [...]. Not verified: [...]."

Source/test drift check. For coding, fix, debug, investigation, test, or review work: when source behavior changes, inspect affected unit/integration/E2E tests and decide from evidence whether tests should change to match intended behavior or the source change is an unintended bug to fix. Do not write tests for migration code; schema/data migrations are one-time execution paths, not core application logic.

Test-Failure Fault Adjudication — When a test fails (or you are debugging or fixing a failure), the job is to determine who is at fault — the source code or the test code. Getting that verdict right matters more than turning the suite green. Binds every debug / fix / test skill identically.

1. Provisional verdict before touching either side. Classify the observed evidence as SOURCE-WRONG, TEST-WRONG, TEST-NOT-OPTIMAL, ENVIRONMENT-BLOCKED, or AMBIGUOUS; then /debug-investigate and trace end-to-start before editing. A green-again suite is NOT the goal.
2. Triangulate against the spec AND the source. If a governing Feature Spec covers the behavior (e.g. docs/specs/** — §3 ACs / §4 BRs / §5 invariants / §8 TCs), it is the tiebreaker for intended behavior — compare BOTH the production source and the failing test against it. With no spec, the documented intent / acceptance criteria / caller contract is the reference. Decide from this evidence whether the SOURCE is wrong or the TEST is wrong.
3. Classify who is at fault, then fix the wrong side at its root:
   • SOURCE-WRONG — production code violates the spec's intended behavior or a clear invariant → fix the source at the owning layer; keep or strengthen the test that caught it.
   • TEST-WRONG — the test encodes a stale or incorrect assertion, setup, or expectation that contradicts intended behavior → fix the test at its root. NEVER weaken an assertion, add a skip, or relax a timeout to force green.
   • TEST-NOT-OPTIMAL — intended behavior is valid but the test seam, timing, or assertion signal is fragile → improve the test without weakening the invariant.
   • ENVIRONMENT-BLOCKED — infrastructure or external state prevents a source/test verdict → preserve diagnostics and stop mutation until the environment is healthy.
   • AMBIGUOUS — evidence or intended behavior does not safely select an owner → ask the user or canonical owner before editing.
   • NEVER change a test to match broken source, and NEVER change source to satisfy a broken test. (Migration code excluded — schema/data migrations are one-time execution paths, not core application logic.)
4. Ask the user when intended behavior is unclear. If no spec covers the behavior, the spec is silent, or the spec is ambiguous about which side is correct, STOP and AskUserQuestion (or consult the canonical spec owner) before editing either side — never silently pick source or test just to make the suite pass.

Reconcile to intended behavior, never to whichever side currently passes — green can encode the very bug.

Read-only/report-only role boundary: when this block is carried by a report-only role (code-reviewer, quality-gate-review, spec-compliance-reviewer, tester, and any other agent whose definition declares it never edits source), "fix the wrong side" means RETURN the adjudicated verdict and the proposed repair to the parent — do not modify source, tests, generated carriers, or user data. The adjudication is the deliverable; the edit is the caller's. Without this sentence the block's step-3 imperatives read as write authority and directly contradict those agents' own declarations (e.g. tester.md "NEVER implement fixes"), which is the sibling SYNC:double-round-trip-review boundary applied to the same class of carrier.

AI Mistake Prevention — Failure modes to avoid on every task:

Re-read files after context changes. Context compaction, resume, or long-running work can make memory stale; verify current files before acting. Verify generated content against source evidence. AI hallucinates APIs, names, claims, and document facts. Check the relevant source before documenting or referencing. Check downstream references before deleting or renaming. Removing an artifact can stale docs, generated mirrors, configs, and callers; map references first. Trace the full impact chain after edits. Changing a definition can miss derived outputs and consumers. Follow the affected chain before declaring done. Verify ALL affected outputs, not just the first. One green check is not all green checks; validate every output surface the change can affect. Assume existing values are intentional — ask WHY before changing OR flagging one as a defect. Before changing or reporting a constant, limit, flag, cutoff, wording, or pattern, read nearby context and history, the CALLER's ordering, and 2+ sibling call sites of the same convention. A doc stating WHAT without WHY is missing rationale, not proof of a missing guard. Surface ambiguity before acting — don't pick silently. Multiple valid interpretations require an explicit question or stated assumption with risk. Assert the outcome your system owns, not the intermediate state your infrastructure owns. When verifying async work, assert the final business state — never the delivery/retry bookkeeping held in shared infrastructure that any co-running process can write. Such a check passes when run alone and flakes the moment anything else shares that infrastructure. Keep shared guidance role-relevant. Universal guidance must help every receiving skill or agent; code-specific obligations belong only in code-specific protocols.

IMPORTANT MUST ATTENTION cite file:line evidence for every claim (confidence >80% to act). NEVER speculate without proof.

MUST ATTENTION apply critical + sequential thinking — every claim needs appropriate traced evidence (file:line for repo/code claims; source URL or artifact section for research, product, content, and docs claims); confidence >80% to act, <60% DO NOT recommend. Anti-hallucination: never present guess as fact, admit uncertainty freely, cross-reference independently, stay skeptical of own confidence.

MUST ATTENTION apply AI mistake prevention — verify generated content against evidence, trace downstream references before deleting or renaming, verify all affected outputs, re-read files after context loss, and surface ambiguity before acting.

Project Protocol Overlay — Before executing this skill, resolve any PROJECT overlay rules layered onto it: match this skill's name against the Target column of the project's skill-protocol index (docs/project-reference/skill-protocols-reference.md by default; a referenceDocs entry in docs/project-config.json overrides the path), taking the most specific matching tier ONLY — exact name > glob > *. That precedence orders overlays against EACH OTHER, never against this skill. Read ONLY the matched bodies, resolved as <protocols-dir>/<Name>.md; a row's Body link is display text, never a read path. A matched body that is missing or malformed is REPORTED and skipped — never reconstructed from the index Description. No index, or no match -> proceed with no overlay, silently. Full contract: .claude/skills/project-skill-protocol/references/registry.md.

Overlays are ADDITIVE ONLY: they ADD rules on top of this skill's own protocol and NEVER replace, override, disable, or reinterpret a rule it already states — removing every overlay must return this skill to exactly its documented behavior. An overlay is a BRIEF, not an authority escalation: it can NEVER waive a workflow gate, git discipline, a review gate, or a user-confirmation gate. A genuine overlay-vs-skill conflict, or two equally-specific overlays that directly contradict -> surface both to the user; NEVER resolve silently.

MUST ATTENTION resolve project protocol overlays for this skill BEFORE executing — most specific matching tier only (exact > glob > *, which ranks overlays against each other, NEVER against this skill), read only matched bodies at <protocols-dir>/<Name>.md; a missing or malformed body is reported, never reconstructed. Overlays are ADDITIVE ONLY (they never replace this skill's own rules) and are a brief, NEVER an authority escalation; an equal-specificity contradiction goes to the user.

Closing Reminders

IMPORTANT MUST ATTENTION Goal: Test local web applications using Python Playwright scripts with server lifecycle management.

Protocols in force (concise digest of the SYNC/shared blocks this skill carries):

• Critical Thinking: critical + sequential thinking; every claim traced, confidence >80% to act.
• Evidence: cite file:line for every claim; never speculate without proof.
• Source/Test Drift: when source changes, inspect affected tests; decide fix vs intended-behavior from evidence.
• AI Mistake Prevention: verify generated content against evidence, trace downstream references, verify all affected outputs, re-read after context loss, surface ambiguity.

IMPORTANT MUST ATTENTION break work into small todo tasks using TaskCreate BEFORE starting IMPORTANT MUST ATTENTION search codebase for 3+ similar patterns before creating new code IMPORTANT MUST ATTENTION cite file:line evidence for every claim (confidence >80% to act) IMPORTANT MUST ATTENTION add a final review todo task to verify work quality MANDATORY IMPORTANT MUST ATTENTION READ the following files before starting:

IMPORTANT MUST ATTENTION READ CLAUDE.md before starting

[TASK-PLANNING] Before acting, analyze task scope and systematically break it into small todo tasks and sub-tasks using TaskCreate.