Artifact Testing

Artifact testing

Publishing distributes. This is the gate before that, and "it rendered" is not a passing result — a page can render perfectly and still be wrong in the other theme, unreadable when printed, unusable by keyboard, and blank for anyone without the connector.

Work the passes in order. Each one catches a different class of failure.

Pass 1 — self-contained

The strict CSP blocks every external host, so anything remote silently fails to load.

• No <script src="http…">, no <link href="http…">, no @import url(http…)
• No remote images; assets are inline SVG or data: URIs
• No fetch/XHR/WebSocket to any origin (connector calls go through the capability API, not through fetch)
• No web fonts — a system stack, or a base64-embedded font you have budgeted for

grep -nE "https?://[^\"')]+" page.html | grep -vE "^\s*<!--|schema\.org|w3\.org"

Every surviving hit is either a link the user clicks (fine) or a resource that will not load (not fine).

• <title> present in the first 8KB — a large inline <style> can push it out of range
• Rendered size under 16MB, and realistically far under

Pass 2 — all three theme states

Not two. Explicit dark, explicit light, and system default which stamps no attribute.

In each, check specifically: body background (a transparent body borrows the host's), SVG text fill (uses fill, not color — the classic silent miss), grid and axis lines, borders, muted text contrast, and disabled/placeholder states.

document.documentElement.setAttribute('data-theme', 'dark');   // then 'light', then remove

Pass 3 — greyscale and print

Board packs get printed. This is where colour-only encoding finally fails visibly.

• Apply filter: grayscale(1) — every series still distinguishable?
• Print preview: controls hidden, charts not split across pages, table headers repeat
• Nothing relies on a background colour that print drops

document.body.style.filter = 'grayscale(1)';   // remember to remove it

Pass 4 — responsive

• 375px (phone), 768px (tablet), 1280px, 1920px
• The page body never scrolls horizontally — wide content scrolls inside its own overflow-x: auto container
• Axis labels still readable at the narrowest width (reduce tick count rather than scaling the chart down)
• Tap targets at least 24×24px
• Usable at 200% zoom

Pass 5 — every data state

The pass people skip, and the source of the most embarrassing failures. Force each one:

• Loading — skeleton matches the final layout's dimensions, no jump when data lands
• Ready — the happy path
• Genuine empty — says what would produce data; does not render an empty chart frame
• Filtered to nothing — a different message, naming the filters, with a reset
• Partial — shows what is missing and why
• Stale — data plus its age plus a retry
• Error / no access — what failed, what it means, what to do; no raw error strings
• Capability absent — claude.use() returned null; the snapshot renders and the page says so

That last one is the one to force deliberately:

const mcp = null;   // temporarily stub it and confirm the page is still useful

Pass 6 — connector calls

If the page declares the mcp capability:

• A real request/response pair was observed for every tool, in this session. Argument names and result encoding are never guessed — the type definitions cover the envelope only
• Observed values were used to learn the shape and then discarded; no real data became placeholder content
• server matches the connector segment exactly, case included; tools uses upstream names
• The manifest is minimal
• Error branches are handled by code: retry only what is retryable, drop data on an authorization denial
• Freshness indicator is driven by the cache timestamp and actually updates

If a pair could not be observed safely, say so in your reply at publish time — not in a note inside the page.

Pass 7 — keyboard and screen reader

• Tab through everything: visible focus ring, logical order, nothing unreachable
• Escape closes and clears
• Large groups (legends, tabs) are one tab stop with arrow navigation
• Every chart has <title> and a <desc> that states an actual takeaway
• Decorative SVG is aria-hidden="true"
• A data-table fallback exists
• Contrast passes in both themes, including muted text and axis ticks
• Reduced motion honoured, and the end state is correct with animation removed
• No value is available only on hover

Pass 8 — the numbers themselves

Rendering correctness is not data correctness.

• Every figure traces to a source, and the page shows the as-of and period status
• Bridges foot — bars sum to the closing anchor (chartkit.waterfallLayout reports this; check footing.ok and fail loudly rather than plugging it)
• Totals in tables equal the sum of their rows
• Percentages are labelled as points or percent
• No number shows more precision than it has
• Suppressed cells are marked as suppressed, not left blank
• Units are pinned per axis and per column

Pass 9 — disclosure

• No customer-identifying detail below the aggregation threshold
• No compensation detail, account numbers, tax identifiers, or credentials
• Benchmarks carry source, vintage, and the stage they apply to
• Estimates and unaudited figures are labelled at the point of use
• Publishing is a deliberate step. Artifacts start private; sharing is the user's decision

The five-minute version

If you do nothing else: greyscale, system-dark, 375px, empty state, tab through it. Those five catch most of what actually ships broken.

Related skills

• artifact-architecture — the constraints being verified
• artifact-theming — the three-state model
• artifact-accessibility — passes 3 and 7 in depth
• live-data-artifacts — pass 6
• app-interaction-patterns — pass 5