QA

stardust:qa — read-only site QA sweep

One live URL in. One evidence-bound findings report out. This skill never edits anything — not site content, not DA documents, not repo code. Its only writes are report artifacts under stardust/qa/. If the user wants findings fixed, that is a separate, explicit follow-up outside this skill.

qa is the post-deploy counterpart of rollout's delivery verification: where rollout asks "did every page ship?", qa asks "is everything that shipped actually correct?" — at all three layers a deploy can silently break:

1. delivery — what the pipeline serves (.plain.html, full HTML, sheets, sitemap)
2. rendered — what a browser shows after block decoration
3. regression — what changed since the last approved state (visual baselines)

A green upper layer never implies the lower one: a publish 200 ≠ delivered, delivered HTML ≠ rendered correctly.

Setup

1. Run the master skill's setup (skills/stardust/SKILL.md § Setup) if not already done this session. qa works standalone too — it only needs a live base URL.
2. Resolve the base URL (the *.aem.live host or production domain). If the user didn't give one, look in stardust/rollout/rollout.json (site.liveHost) or ask.
3. Resolve the inventory source — what pages the sweep covers, merged from any of: stardust/template-map.json (also supplies template assignments for conformance), a paths file, and the live sitemap.xml (always fetched; parity mismatches become findings, so a wrong sitemap can't silently shrink coverage).
4. Optional inputs that unlock deeper checks:
   • --scrape stardust/scrape — verbatim fidelity vs the extraction capture
   • --expected-blocks <json> — explicit per-template block expectations
   • --parity <json> — dynamic parity file to replay (default stardust/dynamics/parity.json; the dynamics check reports parity-missing when absent)
   • --auth-header "token …" / --token-env SITE_TOKEN — protected origins; the secret is sent to the base origin only (otherwise derived by fleet consensus)
   • --blocks-dir <dir> — the site's blocks/ checkout, so the editability check can honour @ew-exempt JSDoc tags (otherwise pass --ew-exempt a,b for index-driven blocks)
5. Browser checks need playwright resolvable from the project (node_modules/playwright). If missing, run the delivery-layer checks only (--checks routing,content,templates,metadata,links) and tell the user what was skipped.
6. Append a phase-transition line to stardust/status.jsonl per reference/run-status.md (master skill) at sweep start/end.

Procedure

Phase 1 — deterministic sweep

node <plugin>/skills/qa/scripts/qa.mjs \
  --base https://main--<site>--<org>.aem.live \
  --template-map stardust/template-map.json \
  --scrape stardust/scrape

Writes stardust/qa/inventory.json, report.json, report.html, screenshots under stardust/qa/shots/, and (first run) visual baselines under stardust/qa/baselines/. Exit 0 = no active errors, 1 = active errors, 2 = infra failure. reference/checks.md documents every check, its finding ids, and severity rationale. Useful variants: --checks <subset>, --max-pages <n> (smoke run), --fail-on warn (strict gate). The ai-readability check reproduces Adobe's AI Content Visibility Checker per page (served words ÷ rendered words) and attributes the gap per block (deploy/reference/ai-readability.md).

The editability check is the post-deploy Experience Workspace editability gate (deploy SKILL.md § 8, EW1–EW10): per page it re-creates the da.live canvas's instrumentation on the served document, lets the live page decorate, and counts which authored texts still carry their editor index. editability/dead-text (error) = a block rebuilt authored text and the author cannot click it in the canvas; editability/duplicated-index (warn) = a presentational clone kept the index. Dead texts inside blocks declared @ew-exempt (or listed in --ew-exempt) are info, not errors.

First run on a site: expect a wave of visual/baseline-created info findings — that is the baseline being established, not a defect. Baselines are screenshots and therefore local (stardust/.gitignore excludes qa/baselines/ and qa/shots/, master skill § Artifacts): later runs on the same machine diff against them; a fresh clone re-establishes them on its first sweep. qa/allowlist.json is the tracked record of judgement.

Phase 2 — triage the ambiguous flags (LLM judgment, still read-only)

The deterministic sweep marks two finding classes as needs triage; read report.json and judge only those:

• content/verbatim-below-threshold — inspect evidence.missingNodes against the live page and the scrape capture: is copy actually lost/corrupted (defect) or acceptably transformed (candidate for the allowlist)?
• visual/visual-diff — open evidence.baseline and evidence.current side by side (they are PNGs; view them): real layout/style regression, or benign dynamism (carousel frame, loaded font, live embed)? Use the bands evidence to locate the changed region.

Record each verdict by annotating the finding in your summary to the user (defect vs non-defect + why). Do not edit report.json scores and do not fix anything.

Phase 3 — report to the user

Summarize: totals by severity, the confirmed defects first (with page paths and one-line evidence), then triaged-away flags with their rationale, then notable warns. Point at stardust/qa/report.html. Recommend — but do not apply — fixes.

Allowlist workflow (documented non-defects)

stardust/qa/allowlist.json (schema in schemas/qa-allowlist.schema.json) keeps known non-defects from drowning every future run — e.g. a source page that itself ships placeholder copy, or a form endpoint deliberately awaiting a client credential. Entries match on check/id/path/messagePattern and must carry a reason. Allowlisted findings stay in the report, greyed out, so the evidence is never deleted.

Only add an entry when the user confirms the flag is a non-defect (or it is already documented as one in the project's records). Never allowlist to make a run green.

Read-only contract

• Writes only under stardust/qa/ (plus the status.jsonl ledger line).
• Never invokes deploy/publish APIs, never PUTs to DA, never edits blocks, styles, or content — even for "trivial" fixes the sweep itself surfaced.
• Reports failure honestly: a crashed check appears in the report as <check>/check-crashed (error), never silently dropped.

Scheduling / CI

The runner is plain node with no plugin-runtime dependency, so the same command works from a GitHub Action or cron for drift monitoring; --fail-on sets the gate. In CI without playwright, pin --checks to the delivery-layer subset.