Verification

Verification

A claim of done names a rung on a delivery ladder, and the evidence in hand has to reach that rung. The failures this skill prevents all came from real sessions: a deployment check reported as correctness verification, a reachable but different environment used as a substitute for the one a bug named, only the internal path tested while users arrive through the external gateway, push success read as took effect, a worker's passing exit code read as acceptance. Lock the target, collect evidence at the rung being claimed, and report the rung actually reached.

Lock the target before collecting any evidence (hard gate)

• Print the target triple — baseline branch or build, environment, entry address — and the source of each: the bug's environment tag, the user's words, the project profile, or the deployed build id. When the source is a bug's tag or the user's words, that environment is the only acceptable one.
• A reachable but non-equivalent environment is never a substitute. When the named environment is down or unauthenticated, report the blocker and stop; anything measured on a substitute is discarded, never uploaded or written back. Case: a UI fix for a bug tagged with one QA environment was checked on a different reachable site after the named one failed its health check; the user rejected it and every result was thrown away.
• Confirm the change under test is in the target: commit SHA or ancestry, image tag, or file hash. A version endpoint's release string or a release name does not prove the commit is deployed.
• When a fixture, address, or case points at a target that has moved, it is a wrong case to delete, not a failure to rerun. Case: probes kept running against a router address retired months earlier, producing false failures until the cases were removed.
• Do not disturb what is in live use. When the environment, the process, or the agent under test is in the user's hands right now, design the check read-only, or state the impact and wait for a go-ahead.

The delivery ladder

Report the highest rung actually reached and name the next one; lower-rung evidence never stands in for a higher rung. Push success is not deployment success, deployment success is not correctness, and correctness on the internal path is not acceptance.

Verify behaviour with real inputs and an old-new comparison

1. Take the inputs from the system's own record — access logs, recent requests, real tickets — and pick the highest-frequency shapes plus the boundary the change touches; invented samples miss the shape production actually sends.
2. Exercise the changed path with those inputs on the new build.
3. Run the same inputs against the pre-change build — the previous environment, the previous image, or the change stashed — and diff the outputs.
4. Before attributing a difference to code, rule out data differences between the two sides: does the artifact, row, or fixture exist on both?
5. Include the failure branch: the input that must now be rejected, skipped, or retried, and confirm it is.

Case: a version-decision service was declared verified because eight removed config keys were absent from the deployed package and the pod was Running. Asked whether that was really all the verification amounted to, the check was redone — the top request shapes were pulled from the access log and the API was called with them on both builds — and one platform returned empty on the new build, traced to missing data in that environment rather than to the code.

Verify through the consumer's real path

• Users reach the service through the external domain, the gateway, and the client; verify there, not only from inside the pod or on localhost. A path that works internally and returns 404 externally has failed.
• Run the check under the affected identity, role, or permission; a privileged account never reproduces an identity-scoped failure.
• Keep every artifact handed over as evidence — a test upload, a probe file — until the consumer has confirmed; cleaning it up early turns your evidence into their false alarm. Case: a test file was deleted right after an internal check passed, the user then opened its external URL, saw 404, and reported a defect that did not exist. Placement and eventual cleanup of such artifacts belong to artifact-hygiene.

Evidence from delegated work

• A worker's status line, its screenshot file, its "tests pass", or a passing exit code is a claim, not acceptance. Re-run the decisive check, read the diff, and drive the surface yourself.
• The acceptance criterion for delegated implementation is that the end user can use the result: it builds, it is packaged or downloadable where the user expects it, and the usage note exists. Case: a delegated implementation stopped at "server capability done", and the user had to ask for the CLI, the binaries, the download page, and the build script one by one.
• Sub-agents that audit stay read-only; after they finish, git status and git ls-remote confirm nothing was committed or pushed under your identity.

Browser and rendered-surface evidence

1. Define the UI surface.
   • Identify the route, the state, the interaction, the data boundary, the target viewports and breakpoints, and the user-visible success criteria the check has to demonstrate.
   • Validate dynamic or external data at its boundaries, so a missing or malformed feed is read as a data defect rather than a layout one.
2. Choose the browser tool by intent.
   • Use Playwright for reproducible assertions, regression coverage, route/network mocking, and CI-stable automation.
   • Use Chrome DevTools for one-off visual inspection, layout/scroll diagnosis, performance traces, Core Web Vitals, request waterfalls, cache/timing analysis, memory, and low-level WebSocket inspection.
   • Use a session-aware browser when the task needs navigation, auth, forms, scraping, or profile/session reuse.
   • For locator design, web-first assertions, storageState auth, mocking boundaries, isolation, and flaky-test triage, apply references/playwright-essentials.md; for a large, long-lived spec suite (deep Page Object Model, sharding, component/Electron testing), follow the project's existing Playwright conventions.
   • When a Playwright run fails before it reaches the app because the runner, browser, download, or media helper cannot start, apply references/playwright-toolchain.md instead of reading that failure as UI evidence.
   • Before hand-rolling authentication, captcha-solving, or a login flow, search the project for an existing login or verification harness and reuse it. Projects that need browser verification often already ship a tested one (zero-touch auth entry, slider/image-captcha solver, MFA/TOTP), and reinventing it in DevTools or by vision is brittle and wastes turns. Hand-drive the browser only when the project genuinely has no such helper.
3. Preserve authenticated context deliberately.
   • Do not assume a fresh isolated browser has the required cookies, profile, or permissions.
   • To recover an authenticated session you lost (a dropped connection, an extension seizing the tab), open a new tab in the same profile so the cookies and login carry over; do not relaunch a fresh or isolated context, which discards the auth. Confirm with the user before any relaunch that could touch account state.
   • A blank or error render is a load failure, not a result: never read a prior mutating action's success or failure off it, and re-verify the outcome from an independent record before retrying or reporting.
   • When a mutating flow is interrupted, assume the server-side record (a case, order, or ticket) may already exist: re-query it and resume by its id rather than restarting the flow, which risks a duplicate submission.
   • If a browser tool cannot start, connect, or attach, report that tool failure before falling back.
   • When the user has removed the blocker — logged you in, granted access, opened the tool for you — carry the real interaction through to completion and capture the evidence (the live run, its screenshots, or the recording); do not stall or substitute a written description for the actual interaction the task called for.
4. Do not replace regression coverage with inspection. DevTools and session browsers diagnose and execute; a durable user-visible bug needs the smallest relevant Playwright or project-native check unless the user asked only for an ad hoc look.
5. Verify interactive HTML before done.
   • Clear the thresholds a rendered page has to meet: type still readable at 200% zoom, contrast of at least 4.5:1 for body text and 3:1 for large text and chart elements, no horizontal scrolling at a 320px width, adequate hit areas, and accessible modal and focus behaviour.
   • For exploratory master-detail interfaces, drive a repeated inspect-return-inspect flow and verify that opening detail does not discard task-relevant exploration state or leave keyboard focus stranded.
   • Load the rendered output, check console errors, key elements, and interactions, and capture a screenshot of the decisive state.
   • When re-verifying a local file you just edited, hard-reload with the cache disabled; otherwise a stale cache shows the previous version and you verify the edit you did not make.
   • For anything rendered into a fixed frame (slides, cards, fixed-aspect canvases), check every instance for overflow and clipping at the target size — content that fits while drafting silently overflows the frame, and the clipped part is invisible to its author.
   • For a deck or page that will be projected, verify at OS-level full screen at the venue's aspect ratio (16:9) — a browser window's inline "full screen" is not the venue's — stepping through every page for leftover margins, overflow, and broken layout; when the venue is dark, also check for glare-bright text that dazzles in a dark room.
   • When playable media sits inside a click-to-dismiss overlay, stop the media's own clicks from reaching the dismiss handler and pause playback on close; otherwise the first click on the controls dismisses the overlay, and a dismissed overlay keeps playing.
   • Where a container binds global keys or screen zones to navigation, decide explicitly which keys belong to focused media, then drive it: confirm that clicking content does not navigate, and that a focused player or embedded frame has not swallowed the keys the reader needs.
6. Capture what is actually rendered.
   • Settle animations before capturing: force or wait out entrance, staggered, and scroll-triggered motion, or the capture shows partial content and stands as false evidence of the rendered layout. Wait for images to finish decoding too, because a capture taken mid-decode shows blank or black regions.
   • Treat a defect that appears only in such an early capture as a phantom: re-capture and confirm it is real before editing the page, or you will "fix" a correct page against a false screenshot.
   • Read what is on screen from the pixels, not from the log, transcript, or DOM text that produced them — a UI renders streamed content progressively, so a timestamp marks when a message completed, not when it first became visible. When it matters what a viewer could see and when, extract frames and check them.
   • State what a capture does not cover before handing it over as evidence.

Report

• State the rung reached, the environment and build, the inputs used, and what stays unverified and why.
• Say what a capture or a run does not cover: a screenshot proves one state at one size, not the flow around it, and a recording of a passing run is not a regression test.

Stop conditions

• The named environment cannot be reached in the right auth or permission state — a blocker, not a licence to substitute.
• The change is not in the deployed build.
• The evidence in hand belongs to a lower rung than the claim being made.
• The screenshot or browser state does not match the claimed user flow.
• A one-off visual check is being offered as durable regression coverage.
• A login or captcha flow is being hand-implemented while the project ships a helper for it.
• The task is only writing or debugging Playwright test code, with no evidence to capture.

reader-facing-writing owns why a visual is needed; technical-diagramming owns a diagram's model and validation; artifact-hygiene owns where evidence files live and when they are cleaned up; change-discipline owns what to change and why before verification starts.