comic-blind-comparison-review — Double-Blind A/B of Two Finished Comics (Phase 1, step S12, post-final)
The "vs baseline" capability the comic side otherwise lacks. The per-panel
comic-director panel_gate and the cross-panel assembly_gate operate
during production on single units; this skill operates after production, on two complete works —
our cross-model-audited progressive comic (authored by the comic-author suite,
baked by comic-director) vs a naive single-prompt one-shot baseline — and
proves the spiral pipeline beats the one-shot, under a blind so neither reviewer can favor the home team.
It is a direct, faithful port of aris_movie's blind-comparison-review from video to a whole-comic A/B
(outputs/aris_movie_progressive.mp4 vs outputs/baseline_sd2_naive.mp4 → two rendered comic page sets), and
it is an evaluation skill, not an authoring one: it never edits a comic, it judges two finished ones.
Cardinal lesson baked in as a gate, not prose: the order of operations IS the integrity guarantee. A single coin-flip seals
.ab_mapping.jsonandchmod 600s it before any reviewer is invoked; the seal's mtime MUST precede both reviews; from seal-time until UNSEAL the orchestrator (Claude) may not read or quote the mapping; any home-team token in a reviewer prompt is a HARD-ABORT, not a warning. If the orchestrator reads the mapping before both reviews land, it may subconsciously phrase the synthesis to favor the home team — so the timestamp ordering is the audit trail, and "blinding integrity" is REPORTED, never assumed.
progressive comic ┐ ┌─▶ comparison.md (Chinese deliverable)
(our audited) ├─▶ ⓪ SEAL .ab_mapping.json (1 coin-flip, │
baseline comic ┘ chmod 600, mtime BEFORE any reviewer) │
(naive one-shot) │ ▼ │
│ ① STAGE pages → A/ and B/ (sealed ids) │
│ ▼ │
│ ② CODEX blind review (xhigh, RO) ─────┤ (no home-team token; reads SHARED
│ ▼ │ blind spec = intent + ART_BIBLE only)
│ ③ GEMINI blind review (per-page + ─────┤
│ synthesis, auto-gemini-3) │
│ ▼ │
└─▶ ④ UNSEAL (verify seal_ts < BOTH reviews) ┘
▼
⑤ WRITE comparison.md + ⑥ WIKI A/B verdict nodes + edges
Constants
- ARTIFACTS = two FINAL whole comics.
progressive= our spiral output (comic-director's baked frames + the single-file viewer, orcomic.json→ render);baseline= a naive single-prompt one-shot comic of the SAME story+style (no spiral, no per-panel gate). For the worked example the contract boundary isexamples/comic_m3_audit/comic.json(the structured-IR artifact under test) +examples/comic_m3_audit/ART_BIBLE.md(the SHARED blind spec both reviewers read). - MAPPING_FILE =
outputs/.ab_mapping.json— the seal,chmod 600. Written by ⓪ before any reviewer. - REVIEWERS (two families, the load-bearing diversity) = Codex via a fresh
mcp__codex__codexcall,model_reasoning_effort: xhigh,sandbox: "read-only"(no model pin — the call follows the local codex CLI config; only the bake pins a model, viarun_comic.get_bake_plan()) ‖ Geminiauto-gemini-3(per-pagemcp__gemini__analyzeFile+ a text-onlymcp__gemini-cli__ask-geminisynthesis). Never downgrade the effort tier (reviewer-routing). Optionally fold in the project's CC-narrative vote to make it the same tri-reviewer panel aspanel_gate(CC ‖ Gemini ‖ Codex) — see Adaptation. - BLIND TOKENS = the only identifiers a reviewer ever sees are
comic_A/comic_B(and per-pageA_p01.png/B_p01.png). Banned from every reviewer prompt:progressive,baseline,ARIS,ARIS-Movie,SD2,naive,one-shot,spiral,our system,system under test, and any rhetorical "show that A is better" framing. - A/B RUBRIC DIMENSIONS (comparative, each scored 0–5 for
comic_AANDcomic_B) =story_comprehension, visual_consistency, overlay_readability, editability_traceability, reproducibility, polish.editability_traceabilityis the structural wedge (see the gate below). - NO PASS/FAIL THRESHOLD — this is comparative A-vs-B, not a gate. Each reviewer emits
overall_winner ∈ {comic_A, comic_B, tie}+confidence 0.0–1.0; the consensus is∈ {progressive, baseline, tie, disagree}. - OUTPUTS =
outputs/.ab_mapping.json(sealed, 600);outputs/blind_review_codex_raw.json;outputs/blind_review_gemini_raw.json(+outputs/gemini_perpage.json);comparison.md(the Chinese human deliverable, peroutput-language); the wiki A/B verdict nodes + edges (§ below). Stdout JSON{comparison_decision_id, consensus_winner, codex_winner, gemini_winner, blinding_integrity, comparison_md_path}.
Input contract — two FINISHED comics + ONE shared blind spec
This skill is pure evaluation; it never authors, edits, or re-bakes a comic.
- Two final artifacts, both complete. Refuse (HALT) if either path is missing, if either comic is not a
whole finished work (a half-baked spiral run is not a fair A/B subject), or if both paths are the same
artifact. The progressive comic must be the shippable output (nothing
escalated/needs_human/flaggedpercomic-director's run-report); a non-shippable progressive comic is not eligible for the head-to-head. - One SHARED blind spec read by BOTH reviewers — the story intent (the comic's logline/script, e.g. the
intent_spec'slogline+narrative_beats) + the style bible (ART_BIBLE.md). The spec is the identity-stripped ground truth ("what this comic is supposed to be"); it must NOT name which artifact is the home team. This is the comic mapping of the video skill'sMOVIE_BRIEF.md+style_bible.md. - Reviewer-independence ≠ reviewer-blinding — BOTH apply (the lesson that names this skill). Independence
= the reviewer sees no Claude prose (the
panel_gaterule). Blinding = the reviewer sees no method IDENTITY of which artifact is the home team. The per-panel gate already does independence; this adds the second axis (identity-blind) for the whole-artifact A/B.
Procedure (followable, fail-closed, in strict order)
⓪ SEAL the A/B mapping (MUST complete BEFORE any reviewer)
Inputs (skill args): PROG_PATH = the progressive comic dir/artifact, BASE_PATH = the baseline comic
dir/artifact, PROJECT_DIR = the project root the wiki lives under (where comic.json + wiki/ sit, e.g.
examples/comic_m3_audit). Bind and FAIL FAST first (the "fail fast if either missing or identical" the
prose promises must be ACTUAL CODE — the seal is the load-bearing integrity step and must not crash). Then a
single coin-flip and write the seal — its mtime MUST precede the first reviewer call:
# --- bind the three skill args explicitly (these are THIS skill's inputs) ---
PROG_PATH="$1"; BASE_PATH="$2"; PROJECT_DIR="${3:-.}"
# --- fail fast: non-empty, both exist, and the two artifacts differ (prose promise → real code) ---
[ -n "$PROG_PATH" ] && [ -n "$BASE_PATH" ] || { echo "HALT: PROG_PATH and BASE_PATH are required args"; exit 2; }
[ -e "$PROG_PATH" ] || { echo "HALT: progressive artifact not found: $PROG_PATH"; exit 2; }
[ -e "$BASE_PATH" ] || { echo "HALT: baseline artifact not found: $BASE_PATH"; exit 2; }
[ "$(cd "$(dirname "$PROG_PATH")" && pwd)/$(basename "$PROG_PATH")" != \
"$(cd "$(dirname "$BASE_PATH")" && pwd)/$(basename "$BASE_PATH")" ] \
|| { echo "HALT: the two A/B artifacts are the SAME path — not a fair head-to-head"; exit 2; }
mkdir -p outputs
if [ $(( $(od -An -N2 -tu2 /dev/urandom) % 2 )) -eq 0 ]; then PROG=A; BASE=B; else PROG=B; BASE=A; fi
# pass the two real paths as argv (NOT os.environ — they were never exported) so the heredoc can never KeyError
python3 - "$PROG" "$BASE" "$PROG_PATH" "$BASE_PATH" <<'PY'
import json, sys, datetime, os
prog, base, prog_path, base_path = sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4]
seal = {
"sealed_at": datetime.datetime.now(datetime.timezone.utc).isoformat(),
"comic_A": os.path.abspath(prog_path if prog=="A" else base_path),
"comic_B": os.path.abspath(base_path if prog=="A" else prog_path),
"progressive_label": prog, "baseline_label": base,
}
json.dump(seal, open("outputs/.ab_mapping.json","w"), ensure_ascii=False, indent=2)
PY
chmod 600 outputs/.ab_mapping.json
(The outputs/.ab_mapping.json seal stores os.path.abspath(...) and that is FINE — cli/validate_wiki.py
only scans wiki/nodes/*.json, never the seal file. The abs-path discipline applies to the WIKI NODE payloads
written in ⑥, not to this seal.) From here until ④ UNSEAL the orchestrator MUST NOT read or quote
outputs/.ab_mapping.json. Reviewers see
only comic_A / comic_B. (Procedural, not a technical guarantee — but it produces a paper trail sufficient
for a research artefact, and that trail IS the integrity claim.)
① STAGE pages into A/ and B/ (the video frame-extraction collapses)
The video skill extracted ~10 frames/video into comparison-frames/{A,B}/; for comics this collapses — the
comic pages ARE the discrete units, deterministic renders, so no extraction or regeneration is needed. Stage
each comic's already-rendered page PNGs into the sealed letter-dirs with identity-free labels
(A_pNN.png / B_pNN.png) so reviewers cite by page, never by home-team identity. The premise is
deterministic pages, don't regenerate — so this skill stages existing PNGs and never bakes. Use the
deterministic recipe for whichever input form each side ships as (apply it independently to comic_A's source
and comic_B's source — neither side knows which is which):
Form (a) — a comic.json + WHOLE-PAGE renders (the structured-IR case). The canonical page order is
comic.json pages[] (a LIST, each page = a discrete unit). Stage one page-level render per page
(page.page_image / page.rendered_path — the whole page as the reader sees it: panels + HTML bubbles +
narration), never a raw per-panel panel_attempt.image_path (that drops the bubbles/narration and, on a
multi-panel page, the other panels — see the code's HALT). Know that the SHIPPED reference IR does NOT
qualify: examples/comic_m3_audit/comic.json carries NO page_image/rendered_path on any of its 18 pages
(verified), so stage() HALTs on it by design — Form (a) applies only after a separate whole-page render
step has populated those fields; on the shipped IR (and any comic.json like it) route to Form (b)
(rasterize the viewer). Copy into the sealed dir as one PNG per page,
renumbered A_pNN.png/B_pNN.png in pages[] order (never re-bake):
mkdir -p outputs/comparison-pages/A outputs/comparison-pages/B
stage() { # $1=comic.json FILE or a dir containing one $2=letter (A|B)
python3 - "$1" "$2" <<'PY'
import json, sys, shutil, os
arg, letter = sys.argv[1], sys.argv[2]
# NORMALIZE the artifact path: accept a comic.json file OR a dir holding one (never assume it's a dir).
cpath = arg if (os.path.isfile(arg) and arg.endswith(".json")) else os.path.join(arg, "comic.json")
if not os.path.isfile(cpath): sys.exit(f"HALT: no comic.json at {arg}")
cdir = os.path.dirname(os.path.abspath(cpath))
cj = json.load(open(cpath))
panels = cj["panels"] # dict {"S01": {...}, ...}
pages = cj.get("pages") or [] # list, the canonical page ORDER; each has panel_ids
if not pages: sys.exit(f"HALT: {cpath} has no pages[] — cannot determine page order")
out = os.path.join("outputs/comparison-pages", letter)
def resolve(rel): return rel if os.path.isabs(rel) else os.path.join(cdir, rel)
n = 0
for pg in pages: # pages[] order IS the comic order (don't sort the dict keys)
pids = pg.get("panel_ids") or []
if not pids: sys.exit(f"HALT: page {pg.get('id')} has no panel_ids")
# A fair A/B compares WHOLE PAGES (each panel + its HTML bubbles/narration as the reader sees it). EVERY
# page needs a page-level render — a raw per-panel PNG is NOT a page (it drops bubbles/narration, and on a
# multi-panel page the other panels). No single-panel shortcut: even a 1-panel page is rendered as a page.
src = pg.get("page_image") or pg.get("rendered_path")
if not src:
sys.exit(f"HALT: page {pg.get('id')} has no page-level render (page_image/rendered_path) — rasterize the "
f"viewer to whole-page PNGs (Form b) before A/B; a per-panel PNG is not a page (no regen here)")
src = resolve(src)
if not os.path.isfile(src): sys.exit(f"HALT: missing rendered page image {src} for page {pg.get('id')}")
n += 1; shutil.copyfile(src, os.path.join(out, f"{letter}_p{n:02d}.png"))
print(f"{letter}: staged {n} pages")
PY
}
# DO NOT call stage() blindly — it is Form-(a) ONLY (a comic.json with page-level renders). Dispatch each
# artifact by TYPE via stage_any() (defined after Form (b) below): comic.json → stage(); .html viewer →
# rasterize_viewer(). Both route by the SEALED coin-flip letter ($PROG/$BASE from ⓪), never by reading the seal.
(Sealing caveat — keep the blind intact.) The two stage calls above route each artifact to its sealed
letter via the coin-flip variables $PROG/$BASE from ⓪ — not by reading comic_A/comic_B from the seal
file, which is FORBIDDEN until ④. The orchestrator knows the letters (it flipped the coin) without ever opening
the mapping, so the blind holds; the reviewer still only ever sees A_*/B_* filenames.
Form (b) — a single-file HTML viewer with NO pre-rendered PNGs (the common comic-director ship form). This skill does not own a bake; it is fail-closed by design. Prefer pre-rendered PNGs; if absent, the caller must rasterize first via the repo's own deterministic page renderer, then re-invoke this skill pointing at the PNG dir. If you must rasterize inline, the ONLY sanctioned path is a headless-Chromium screenshot at a FIXED width/DPI (deterministic), one PNG per page, e.g.:
# headless rasterize the single-file viewer → one PNG per page (deterministic fixed window so both sides are
# pixel-comparable). The viewer's per-page anchor is `?p=N` (the comic-director viewer convention).
rasterize_viewer() { # $1=viewer .html $2=letter $3=NPAGES (from comic.json pages[])
CHROME="$(command -v chromium || command -v google-chrome || command -v chromium-browser || true)"
[ -n "$CHROME" ] || { echo "HALT: no headless chromium to rasterize the viewer — supply pre-rendered page PNGs instead"; exit 2; }
local v abs; abs="$(cd "$(dirname "$1")" && pwd)/$(basename "$1")"
mkdir -p "outputs/comparison-pages/$2"
for i in $(seq 1 "$3"); do
pp=$(printf "%02d" "$i")
"$CHROME" --headless --disable-gpu --window-size=1200,1600 \
--screenshot="outputs/comparison-pages/$2/$2_p${pp}.png" "file://${abs}?p=${i}" >/dev/null 2>&1 \
|| { echo "HALT: screenshot of page $i failed"; exit 2; }
done
echo "$2: rasterized $3 pages from the viewer"
}
# e.g. rasterize_viewer "$PROG_PATH" "$PROG" "$NPAGES" (NPAGES = len(comic.json pages[]))
If neither pre-rendered PNGs nor a headless renderer is available → HALT with that message; never feed the reviewers a partial or wrong-DPI rasterize (a corrupted page set makes the A/B score garbage).
Stage dispatch — the actual control flow (picks the form per artifact; never calls stage() blindly).
stage_any() { # $1=artifact (a DIR of pre-rendered *_pNN.png / a comic.json / a single-file .html viewer) $2=sealed letter
case "$1" in
*.html) # the SINGLE-FILE viewer (the common ship form, NO sibling comic.json): rasterize whole-page PNGs.
# derive NPAGES from the viewer's EMBEDDED JSON island (the build inlines comic.json into a <script>).
NP=$(python3 - "$1" <<'PY'
import json, re, sys
html = open(sys.argv[1], encoding="utf-8").read()
n = ""
for blob in re.findall(r'<script[^>]*>\s*(\{.*?\})\s*</script>', html, re.S): # the inlined comic-IR island
try:
d = json.loads(blob)
if isinstance(d.get("pages"), list): n = len(d["pages"]); break
except Exception:
pass
print(n)
PY
)
[ -n "$NP" ] || { echo "HALT: cannot read page count from the viewer's embedded JSON island in $1"; exit 2; } ;
rasterize_viewer "$1" "$2" "$NP" ;;
*)
if [ -d "$1" ] && ls "$1"/*_p[0-9]*.png >/dev/null 2>&1; then # a DIR of pre-rendered whole-page PNGs
mkdir -p "outputs/comparison-pages/$2"; i=0
for f in $(ls "$1"/*_p[0-9]*.png | sort); do i=$((i+1)); cp "$f" "outputs/comparison-pages/$2/$2_p$(printf '%02d' "$i").png"; done
echo "$2: staged $i pre-rendered page PNGs"
else
stage "$1" "$2" # a comic.json file or a project dir — stage() normalizes + requires page-level renders
fi ;;
esac
}
stage_any "$PROG_PATH" "$PROG" # route by the SEALED letter; no read of the seal file → blind holds until ④
stage_any "$BASE_PATH" "$BASE"
Page-count parity gate (fail-closed). After staging both sides, assert A and B staged the same number of pages — an unequal A/B is not a fair head-to-head:
nA=$(ls outputs/comparison-pages/A/*.png 2>/dev/null | wc -l | tr -d ' ')
nB=$(ls outputs/comparison-pages/B/*.png 2>/dev/null | wc -l | tr -d ' ')
[ "$nA" -gt 0 ] && [ "$nA" = "$nB" ] || { echo "HALT: page-count parity failed (A=$nA B=$nB) — not a fair A/B"; exit 2; }
Page filenames embed only the sealed letter + a page index (A_p03.png) so reviewers cite by page, never by
home-team identity.
② CODEX blind review (read-only, xhigh)
A fresh mcp__codex__codex call (NOT codex-reply), config {"model_reasoning_effort":"xhigh"},
sandbox: "read-only". It sees the two page dirs (comparison-pages/A, comparison-pages/B) + the SHARED
blind spec (the identity-stripped intent + ART_BIBLE.md). Scrub the prompt with the banned-token scan
(Constants) BEFORE submission. The prompt asks Codex to read the pages itself and score each comic 0–5 on
every A/B rubric dimension, cite page filenames as evidence, and output JSON only. Save verbatim to
outputs/blind_review_codex_raw.json. Required JSON shape:
{"comic_A": {"<dim>": {"score": 0-5, "evidence": "...cites A_p0X.png..."}, ...},
"comic_B": {"<dim>": {"score": 0-5, "evidence": "...cites B_p0X.png..."}, ...},
"overall_winner": "comic_A|comic_B|tie", "confidence": 0.0-1.0, "rationale": "..."}
If Codex returns non-JSON → retry once, stricter; still non-JSON → write a {"parse_failed": true} stub
and continue Gemini-only, noting the degradation in comparison.md.
③ GEMINI blind review (independent, same blind rules)
3.1 per-page — mcp__gemini__analyzeFile, model auto-gemini-3, one call per image (every page in A/
and B/), each page NOT seeing the other comic. Per-page JSON
{label, page_filename, character_appearance ≤150, scene_signature ≤80, style_features ≤80, artifact_severity_0_to_5 (int, 0=clean 5=catastrophic), notable_anomalies[]}; persist to
outputs/gemini_perpage.json.
3.2 synthesis — mcp__gemini-cli__ask-gemini (text-only), fed both per-page observation lists + the blind
spec summaries (≤1000 chars each), scoring the same 0–5 A/B rubric, JSON only. Save verbatim to
outputs/blind_review_gemini_raw.json. Gemini is always auto-gemini-3. Fail fast if fewer than the staged
pages succeed per comic (the video floor was <6 frames/video → abort; for comics use "every staged page must
return, else re-run that page once then abort").
④ UNSEAL — only NOW read the mapping
Read outputs/.ab_mapping.json for the first time. Verify the seal predates BOTH reviews (its mtime is
older than both raw review files):
# portable mtime probe — BSD stat is `-f %m`, GNU stat is `-c %Y`; python3 works identically on both:
mt() { python3 -c 'import os,sys; print(int(os.path.getmtime(sys.argv[1])))' "$1"; }
SEAL=$(mt outputs/.ab_mapping.json)
CODEX=$(mt outputs/blind_review_codex_raw.json)
GEM=$(mt outputs/blind_review_gemini_raw.json)
[ "$SEAL" -lt "$CODEX" ] && [ "$SEAL" -lt "$GEM" ] && echo "intact" || echo "compromised"
If out of order → blinding_integrity = "compromised": still produce comparison.md, but flag it
prominently, never silently elide. Re-label comic_A/comic_B back to progressive/baseline in the
PARSED reviews; keep _A/_B in the RAW files for audit (never edit a raw file after it is written).
⑤ WRITE comparison.md (Chinese — per output-language)
Six fixed sections, in Chinese:
- 执行裁定 / Executive verdict — each reviewer's winner + confidence; the
consensus. - 逐维度评分表 / Per-dimension table —
维度 | Progressive (Codex / Gemini) | Baseline (Codex / Gemini) | Δacross all six A/B dimensions. - 证据 / Evidence — verbatim per-dimension evidence + page citations (
A_p0X.pngre-labeled to which artifact it actually was, after unseal). - 可编辑性 / 架构差异表 / Editability table — the wedge made concrete (single-unit regen cost, failure localizability via wiki edges, minimum human-patch unit, audit record). See the worked-example table below.
- 盲评审计 / Blinding audit — mapping path, seal→first-review→last-review time window, leakage yes/no,
blinding_integrity. - 结论 / Conclusion — 3–5 sentences, honestly recording any dimension the baseline won (hiding it defeats the cross-model adversarial check).
⑥ WIKI — the A/B verdict as schema-valid nodes + edges
Write the post-final A/B record per schemas/node_schema.json (§ below), append the edges, and print the
stdout JSON. The reviewers (different model families) — never this orchestrator — produce the verdict; the
orchestrator only seals, stages, unseals, and records (acceptance-gate:
the loop drives, it cannot acquit).
Resolve a REAL target_node_id first (no dangling edges). The slim node_schema.json (v3.0) has no
whole-comic node_type and no top-level comic.json.wiki_node_id — so "the progressive comic's anchor node"
does not exist by default and an edge to it would dangle (cli/validate_wiki.py resolves an endpoint ONLY
to a wiki node_id ∪ a comic.json panels[*].wiki_node_id; an unresolved dst is a hard FAIL). The A/B record
MUST point target_node_id at an endpoint that already resolves. Pick, in priority order:
- A caller-supplied AUTHOR anchor — if the project has an
intent_specnode (node_idintent:<slug>) or anoutline_specnode (node_idoutline:<slug>) for the progressive comic, the caller passes that node_id asTARGET_NODE_IDand the A/B nodes point there (this is the cleanest "the whole comic" anchor; it must be a real node inwiki/nodes/). - Else, the progressive comic's FIRST panel anchor — a
panel:<slug>that exists as acomic.json panels[*].wiki_node_id(e.g. the worked example haspanel:s01_aris_comic_v1). This is the pragmatic fallback the slim schema supports today: it resolves via the comic.json panel-anchor union, so the edges validate. Record in the decision payload that the anchor is the first-panel proxy (the A/B is about the whole comic; the panel is just the schema-valid attachment point). Whichever you choose, the chosen id MUST already be a resolvable endpoint — verify withpython3 cli/validate_wiki.py <PROJECT_DIR>BEFORE declaring done; if it doesn't resolve, the skill HALTs rather than write a dangling edge.
EXACT gate (A/B rubric) — dimensions, scoring, the wedge, vetoes
Ported from the aris_movie blind-comparison-review rubric, adapted from video to comic (the video skill's
narration_sync dimension is dropped and folded into overlay_readability = caption/bubble↔panel
reading-order alignment, which the assembly_gate already scores). Each reviewer scores each comic 0–5 on
all six dimensions — there is no numeric pass threshold; the verdict is the comparative overall_winner
confidence, and the consensus across the two families.
story_comprehension— can the reader follow the through-line end to end (the comic's logline lands) from the pages alone? (video: scene_jumps coherent vs abrupt unexplained jumps.)visual_consistency— character identity (face/clothes/age) + palette/render stable across pages vs drifting. Design-aware: the two-world warm/dark split is by-design, not drift (ART_BIBLE §0.5); a reviewer must only flag miscolored worlds, never the split itself. (video: character_consistency + style_drift.)overlay_readability— bubbles/captions legible, correctly ordered, aligned to the right panel; no garble; bilingual text fits. (replaces the videonarration_syncdimension.)editability_traceability— THE STRUCTURAL WEDGE. Could a human editor patch a bad page without a full re-gen, and is each failure localizable? 5 = a single panel + its local prompt is the minimum patch unit and every panel traces to itspanel_attempt/review/decision/failure_modewiki nodes; 0 = the whole comic is one indivisible bake with no trace, so any defect forces a total re-gen. This is where the pipeline's contribution shows even when the baseline's one-shot output is visually strong — it is an architectural FACT (wiki-backed traceability), not a pixel measurement, so it can WIN the A/B even while the baseline ties or beats onpolish. (video: editability_proxy.)reproducibility— can the artifact be re-derived? (re-run thecomic.json+ content-SVG blueprints + the audited spiral → the same comic) vs a one-shot whose exact output is unrecoverable. (video: deterministic re-render of cached frames.)polish— raw aesthetic finish / artifact floor (the dimension the naive one-shot can plausibly win). (video: artifact_floor / artifact_severity.)
Veto / hard-abort rules (the load-bearing rigor — abort beats taint):
MAPPING_FILEreferenced in ANY reviewer prompt → HARD-ABORT"Blinding compromised — fix prompt template and re-run". Better to abort than ship a tainted A/B.- Any home-team token (
progressive/baseline/ARIS/SD2/naive/one-shot/spiral/our system/system under test, or a rhetorical "show that A is better") in a reviewer prompt → the prompt is REJECTED before submission by the banned-token scan. - Reviewer raw files predate
MAPPING_FILE→blinding_integrity = "compromised", flag prominently incomparison.md, never silently elide. - Codex non-JSON → retry once stricter; else
{"parse_failed": true}stub + continue Gemini-only, noted in the md. - Fewer than the staged pages return from a reviewer → re-run the missing page once, then fail fast.
Invariants (REPORTED, not assumed): seal written before any reviewer AND seal_ts < both review-completion ts; no home-team token in any reviewer prompt; Codex always xhigh, Gemini always auto-gemini-3; raw
responses persisted verbatim, never edited after writing; the two reviewers are different families (agreement
is then non-trivial signal — disagree is a real finding, not noise to reconcile).
Two engine contracts (fail-closed) — why a fair A/B is even possible
This skill judges finished comics, but the reason the progressive side can be re-derived and a baked panel audited is the same two contracts the upstream authoring + spiral enforce — name them so a reviewer of THIS skill knows what "reproducibility" and "editability" are grounded in:
- Every panel needs a
condition.content_svg— the progressive comic is reproducible precisely because each panel was baked from a deterministic SVG blueprint (comic.jsonpanels carrycondition.content_svg, e.g.assets/s01_ddl_anchor_v1.svg); the engine refusescontent_svg: null. A naive one-shot comic has no such blueprint, which is why it scores low onreproducibility. - A baked figure-panel needs
expected_literals— every baked panel declares its salient ASCII tokens (comic.json'scondition.expected_literals, e.g.["DDL","T-24:00:00"],["+6.2"]), which the spiral's double-blind token-diff verified during production; a one-shot bake has no audited literals, which is why its numbers are unverifiable and itseditability_traceabilityis 0. Do not re-run those gates here — this skill only credits the progressive side for having honored them, via the editability table.
Node it reads / writes (schemas/node_schema.json)
Schema note (read this — it is the orphan-panel-class discipline). The aris_movie source describes a single rich
comparisonnode withbaseline_for/compared_againstedges. The SLIM comicnode_schema.json(v3.0) has nocomparisonnode_type and nobaseline_for/compared_againstedge types; the canonical edge form is{src, dst, type}(NEVERfrom/to/edge_type) withtype ∈ {attempt_of, reviews, decides, failure_of, rollback_of, supersedes}. So this post-final A/B is recorded with the schema's existing verdict vocabulary: tworeviewnodes (one per reviewer) + onedecisionnode (the consensus). The rich aris_moviecomparisonpayload (per-dim scores, blinding audit) is carried as the decision node's payload — the schema isadditionalProperties: true, so extra fields validate.
Reads — two finished comics (the progressive comic.json / rendered pages + the baseline rendered pages)
and the SHARED blind spec (the intent_spec's logline/narrative_beats + ART_BIBLE.md). It does not
mutate any upstream authoring node.
Writes (status complete for the reviews, final for the decision — the runtime canon). The exact
PAYLOAD_REQUIRED from cli/validate_wiki.py is review → ["target_node_id","reviewer","gate_kind"] and
decision → ["target_node_id","verdict","gate_kind"]; node_id must match the schema pattern
^(…|review|decision|…):[a-z0-9_-]+$ (lowercase + _/- only — cmp_… is fine, <YYYYMMDD> is digits):
review× 2 (node_idreview:cmp_<reviewer>_<YYYYMMDD>) — payload required (all three):target_node_id(the resolved anchor from ⑥ — the caller'sintent_spec/outline_specid, or the first-panel proxypanel:<slug>; NEVER an id that doesn't resolve),reviewer(codex|gemini),gate_kind("blind_comparison"). Recommended optional fields (validate underadditionalProperties):raw_path(the verbatimblind_review_*_raw.json, project-RELATIVE — see the path rule below),winner(progressive|baseline|tie, after unseal),confidence,per_dim_scores(the unblinded 0–5 per-dimension scores for both artifacts). The raw reviews live in files; the node only summarizes (no separate "review-node per dimension").decision× 1 (node_iddecision:cmp_progressive_vs_baseline_<YYYYMMDD>) — payload required (all three):target_node_id(the SAME resolved anchor),verdict(the consensus"progressive"|"baseline"|"tie"|"disagree"),gate_kind("blind_comparison"). Recommended optional:progressive_path,baseline_path,comparison_md_path(all project-RELATIVE, see below),codex_winner,gemini_winner,anchor_kind("author"|"first_panel_proxy"— record which ⑥ option was used),blinding_audit{sealed_at, first_review_started, last_review_completed, integrity, leakage_reason}. This is the single authoritative A/B record (the aris_moviecomparisonnode's role).
Path fields MUST be project-relative (release-gate rule).
cli/validate_wiki.pyrunsabs_path_leaks()over EVERYwiki/nodes/*.jsonpayload and hard-fails (sys.exit 1) on any string containing/Users/or/home/. The only in-skill source of these paths is the seal, which stored them viaos.path.abspath(...)— so before writing the nodes you MUST relativize:progressive_path,baseline_path,comparison_md_path, and anyraw_path=os.path.relpath(p, PROJECT_DIR)(or just theoutputs/…tail). Add a one-line guard mirroring the validator before writing each node — fail fast rather than emit an invalid node:import json for s in (json.dumps(review_payload), json.dumps(decision_payload)): assert "/Users/" not in s and "/home/" not in s, "HALT: payload has an absolute path; relativize before writing"(The
outputs/.ab_mapping.jsonseal itself keepsos.path.abspathand is fine — the validator never scans it, onlywiki/nodes/.)
Edges (canonical {src, dst, type} only; type ∈ validate_wiki.py EDGE_TYPES). The dst is the
resolved target_node_id from ⑥ (the caller's intent_spec/outline_spec id, or the first-panel proxy
panel:<slug> — an id that already resolves; never the literal string <progressive anchor>):
review:cmp_codex_… --reviews--> <target_node_id> and review:cmp_gemini_… --reviews--> <target_node_id> (each
with the optional reviewer + verdict edge fields); decision:cmp_… --decides--> <target_node_id> (with the
optional verdict field). reviews/decides are both legal EDGE_TYPES and exactly the verbs the example wiki
uses for review→target / decision→target. All verdict edges carry only logical-order created_at, never
wall-clock. After appending, python3 cli/validate_wiki.py <PROJECT_DIR> MUST pass (every dst resolves).
Trace every reviewer prompt+response to trace.jsonl (review-tracing).
Worked example
The canonical exhibits are examples/comic_m3_audit/comic.json
(the structured-IR artifact under test — the progressive side) and
examples/comic_m3_audit/ART_BIBLE.md (the SHARED blind
spec both reviewers read). The concrete pattern to copy:
The progressive side is the structured IR, the baseline is its one-shot foil.
comic.jsonis a 24-panel / 18-page comic-ir/1.0 (the storyboard declared 19 pages; the endcard folds intoP_B12.closing.image, so the compiled IR ships 18pages[]) with, per panel, acondition.content_svgand (for baked panels)condition.expected_literals— e.g.S01 → content_svg: "assets/s01_ddl_anchor_v1.svg", expected_literals: ["DDL","T-24:00:00"];S12 → content_svg: "assets/method_random_vs_schema_first_v1.svg", expected_literals: ["+6.2"]. The baseline for the A/B is a naive single-prompt render of the SAME story+style (no blueprint, no per-panel gate). The reviewers never learn which is which.ART_BIBLE.mdis the SHARED blind spec, and it is identity-free — it declares the convergence target ("style_consistency的字面定义就是符合本文件"), the two-world warm/dark palette (§0.5: "冷暖是 by-design 的,不是漂移" — copy this verbatim into the reviewer'svisual_consistencyinstruction so the reviewer does not false-flag the by-design split), the identity hex-lock (§1), and the env-density taste calibration (§6.5). Feed it to BOTH reviewers; it never says which artifact is the home team.The editability table (comparison.md §4) — the wedge made concrete. Copy this exact shape (filled from the
comic_m3_auditfacts):维度 / dimension Progressive (our structured render) Baseline (naive one-shot) 单元再生成成本 / single-unit regen cost 1 panel + its local content-SVG blueprint + local prompt 整本重生成 / the whole comic 失败可定位 / failure localizable yes — each panel → panel_attempt/review/decision/failure_modewiki nodesno — one indivisible bake, no trace 最小人工补丁单元 / min human-patch unit 1 panel (re-bake just S14) 整本 / the whole comic 数字可审计 / numbers auditable yes — expected_literalstoken-diff'd during the spiral (+6.2→+1.4audit)no — no audited literals 可复现 / reproducible yes — re-run comic.json+ blueprints + spiralno — exact output unrecoverable This is exactly the case where the baseline may tie or win on
polishyet the progressive side wins overall oneditability_traceability+reproducibility— KEEP ≠ final: a structured-but-uglier render can beat a one-shot pretty one, and the comparison must say so honestly.Disagreement is a finding, not noise. If Codex picks the progressive comic and Gemini picks the baseline →
consensus = "disagree", recorded as such in the decision node and §1 ofcomparison.md— a real research result (single-reviewer "we win" could just be that family's bias).
Hard do / don't (earned lessons)
- DO SEAL
.ab_mapping.jsonwith a single coin-flip,chmod 600, before any reviewer; never read or quote it until ④ UNSEAL; verifyseal_ts < both review tsand report the integrity (intact/compromised). - DO scrub every reviewer prompt with the banned-token scan; a
MAPPING_FILEreference is a HARD-ABORT, any home-team token is a pre-submission REJECT. - DO use two different reviewer families (Codex xhigh ‖ Gemini auto-gemini-3); agreement is then real
signal and
disagreeis a finding to record, never to reconcile away. - DO persist raw reviews verbatim and keep
_A/_Bin the raw files for audit; only the PARSED copies get re-labeled to progressive/baseline after unseal. - DO record honestly any dimension the baseline won (especially
polish); hiding it defeats the cross-model adversarial check. - DON'T treat this as the per-panel gate —
panel_gate/assembly_gaterun DURING production on single units; this runs AFTER, on two complete works. It is an evaluation, not authoring. - DON'T regenerate the comic pages for the A/B — they are deterministic renders; stage the existing pages.
- DON'T let the orchestrator (Claude) acquit the winner — the two reviewer families produce the verdict; Claude only seals, stages, unseals, and writes the record. The loop drives, it cannot acquit.
- DON'T invent a
comparisonnode_type orbaseline_for/compared_againstedges — record the A/B with the slim schema'sreview+decisionnodes and canonical{src,dst,type}edges.
Adaptation note (video → image-comic)
What changed from aris_movie's blind-comparison-review: DROP the ffmpeg/ffprobe frame-extraction (comic
pages ARE the discrete deterministic units — stage A_p01.png-style directly); DROP the narration_sync
dimension (folded into overlay_readability = bubble/caption↔panel reading-order alignment, which
assembly_gate already scores); KEEP the other dims and add the comic-native ones the panel_gate uses
(identity consistency under visual_consistency; baked-text fidelity via expected_literals under
overlay_readability). Optionally fold the project's CC-narrative reviewer into the panel so the blind A/B
uses the same tri-reviewer set as production (CC ‖ Gemini ‖ Codex) instead of two. Everything else — the seal,
the blind tokens, the timestamp audit, the unseal, the editability table, and the A/B verdict record — ports
1:1. This is an EVALUATION skill that complements the authoring suite by giving the finished comic a
baseline-relative honest verdict; it does not replace any authoring step.
Protocols (govern
…(truncated)