Bs Check

Confidence Check

Force the assistant to audit its own confidence on a claim before the user acts on it, sends it to a client, or pushes it to production. Designed to catch AI slop - plausible, well-worded output that has not actually been validated against evidence - at the moment it would cost something.

Make it yours (recommended): this skill fires best when its description knows your high-stakes moments. Add your stack (your CMS, workflow engine, cloud provider), your client-facing rituals, and your own trigger phrases to the frontmatter description. Also see the Domain lenses section at the bottom - add a lens per domain you work in.

Subcommand dispatch (read before anything else)

Parse the first word of the invocation arguments:

Lane rule (do not blend lanes): premise and approach audit thinking - right problem? right approach? - and end in a reframe/proceed verdict, never a PASS/BLOCKED gate. The verify pipeline audits claims and gates actions. The Stage 1 judge's pure-judge invariant (score only, never generate alternatives) applies ONLY to the verify pipeline; the premise/approach protocols are generative by design and never feed the judge. If a premise/approach run surfaces a factual claim about live systems, recommend a follow-up verify run - do not merge the two into one report.

Verifier framing (read first)

You are not the author of the claim under review. Read it as someone else's work that you have been asked to audit. You may not defend, expand, or restate the claim more confidently than it was originally made. Your only job here is to score, annotate, and surface gaps. The same reasoning that produced the claim will rationalize it - the only way to avoid that is to switch roles.

If the assistant just produced the claim in this same chat, treat conversational momentum as a bias to actively counter, not as evidence of soundness.

This is a calibration tool, not a replacement for investigation. It does not certify correctness. It tells you whether the claim is currently load-bearing enough to act on.

How this skill runs

The verify pipeline has three execution modes with different orchestration:

Quick mode -> inline self-audit (current thread). The same Claude that produced the claim walks the rubric (Steps 0-9 below) and outputs a one-paragraph verdict. Fast, low friction, but carries known LLM self-preference and conversational-momentum bias. Right for gut-checks mid-investigation.

Full mode -> cross-judge verification pipeline (dispatched). Used when the claim involves public-domain facts (APIs, pricing, public documentation, general knowledge):

1. Stage 1 dispatches a fresh-context Claude subagent (Agent tool) to score the claim against the rubric. The subagent has no memory of this chat - no prior reasoning, no main-thread confidence, no tool history. It returns a strict JSON structure.
2. Stage 2 calls a web judge per Critical evidence-thin sub-claim for web-grounded external verification. The web judge is blind to Stage 1's verdict. Web judge selection (first available wins):
   • Perplexity MCP (mcp__perplexity__perplexity_reason) - preferred: an independent model with live web grounding. If you have the Perplexity MCP connected, Full mode uses it automatically.
   • Built-in web tools (WebSearch / WebFetch) - dispatch a fresh subagent equipped with web tools and the Stage 2 prompt. The subagent receives ONLY the sub-claim text and domain tag, preserving blindness to Stage 1.
   • Neither available - skip Stage 2 with an explicit banner (see Exception handling). Never skip silently.
3. Stage 3 applies a deterministic rule table to merge the two judges and produce a binary blocking gate (PASS / BLOCKED).

Direct-Obs mode -> direct verification against live systems. Used when the claim is primarily about internal infrastructure or proprietary state (your servers, workflow engines, CMS sites, databases, app configs) - domains where the web has no signal and a subagent re-reasoning would just produce more text against the same uncertainty:

1. Main thread runs direct observation commands (Bash, MCP tools, curl, SSH, CLI tools, database queries, etc.) to verify each sub-claim against the live system.
2. Findings are tabulated as Facts with sources.
3. The bs-check rubric is applied inline (Steps 0-9) using Direct-Obs evidence - which the rubric explicitly rates as the highest-quality evidence type (cap 98%).
4. Output is the standard Full mode format with a Direct-Obs banner at top declaring the substitution.
5. Stage 1 subagent + Stage 2 web judge are explicitly bypassed (not silently skipped). The banner says so.

This pipeline is grounded in 2024-2026 multi-agent verification research (self-preference studies; multi-agent failure taxonomies; structural-gate guidance). The two key invariants:

• Context isolation - the Stage 1 subagent sees the claim and the rubric, never the main-thread's chain-of-thought. The Stage 2 web judge sees only the sub-claim text + domain tag, never Stage 1's confidence. In Direct-Obs mode, this invariant is satisfied by modality shift (text claims -> executable verification commands) rather than by context isolation - and modality shift is the stronger of the two patterns per the research.
• Structural gate - Full and Direct-Obs modes both produce a blocking flag, not advice. Downstream action is gated on PASS.

Friction / cost tradeoff:

When to use this skill

Default to Full mode whenever the action is irreversible, client-facing, costs money, or modifies production state. Use Quick mode for in-flight gut-checks where the next step is more thinking, not more action.

Universal high-stakes moments:

• About to tell a client or stakeholder a bug is fixed, an integration works, or a deliverable is ready
• About to push a change to a live production site or service
• About to activate, delete, or modify a production automation or workflow
• About to claim a configuration is correct based on what a dev tool or MCP reported (tool schemas can drift from the live runtime)
• About to assert an API behavior, pricing tier, model capability, or endpoint shape for a third-party service
• About to run a destructive infrastructure command - rm, volume deletion, content deletion, schema migration
• About to send a contract, invoice, or automated message to a client
• About to close a ticket or mark a task "done"
• Mid-investigation, when a working hypothesis is starting to sound like a settled fact
• After the assistant produces a confident-sounding explanation that hasn't been validated against the live system

When NOT to use

• The statement is directly verifiable in this session ("this file exists", "the API returned 200 OK", "the row count is 47") - don't run a confidence chain on a fact you can read off
• Discovery is still wide open and no real claim has been made
• The matter is trivial and reversible with no client or production impact
• The user wants the assistant to do the work (run the query, ssh in, check the actual config) - do that first, then run this on the result

Modes

Mode selection is automatic - see "Pre-flight routing decision" below. The user can override with /bs-check quick, /bs-check full, or /bs-check direct.

Pre-flight routing decision

Before dispatching anything, main-thread Claude must decide which mode to run. This is a lightweight triage of the claim, not a full decomposition. Walk this in order:

Step P1: Explicit mode override

If the user invoked with /bs-check quick, /bs-check full, or /bs-check direct - honor the override. Skip P2. (premise, approach, and fresh never reach this point - they are routed by Subcommand dispatch at the top of this file.)

Step P2: Auto-detect mode

If no override, classify the claim by reading it once and answering:

1. Is the claim primarily about internal / proprietary state? Examples:
   • "Workflow <id> is active / has been tested / has N executions"
   • "Container <name> is healthy / responding / restarted cleanly"
   • "The database error is still happening / is resolved"
   • "This CMS change is safe to push to the client's site"
   • "Env var X is set on the server"
   • "The migration ran successfully on the client database"
   • "The production API keys are deployed in the app's env file"
2. Is the claim primarily about public-domain facts? Examples:
   • "Vendor X's API uses /api/v2 and pricing is $N per million tokens"
   • "Service Y's free tier includes feature A but not feature B"
   • "Model <name> is available on platform Z"
   • "This SaaS automatically schedules weekly payouts for US accounts"
3. Mixed? Example:
   • "Vendor X is cheaper than running our own GPU stack" (public pricing + internal infra)

Step P3: Route

Once routed, jump to the corresponding pipeline section. Do not silently mix paths. If you discover mid-Direct-Obs that a sub-claim actually needs the web judge, finish Direct-Obs and add a ## Late-stage Stage 2 calls section to the output naming which sub-claims were re-routed.

Direct-Obs mode pipeline

Used when the routing decision sends the claim here. Main thread runs verification commands directly against the live system. Subagent + web judge are explicitly bypassed (not silently skipped - the banner declares it).

Pipeline checklist

1. Create TodoWrite items, one per planned verification command (step 3 below). This makes orchestration observable - if a verification gets skipped, the user sees an unchecked todo.

2. Decompose the claim inline using the bs-check rubric (Step 1 + Step 2 of the Process section below). For each sub-claim, tag criticality, evidence_type, and domain. Most sub-claims here will land as internal-system or proprietary.

3. For each Critical sub-claim, plan and run a verification command. Examples by claim type (adapt to your stack):

   Sub-claim type	Verification commands
   "Container / service <name> is healthy / running"	ssh <host> "docker ps --format ..." + curl <health endpoint>
   "Workflow <id> is active / has been tested"	Query the workflow engine's API for the workflow state + recent execution results
   "Env var <X> is set in <file>"	ssh <host> "grep -E '^X=' <file> | sed 's/=.*/=<set>/'" (mask secrets)
   "Database <X> exists / table has rows"	Run the read-only query against the live database
   "Error <msg> is no longer appearing in logs"	ssh <host> "docker logs <c> --since 48h 2>&1 | grep -cE '<pattern>'"
   "CMS / page-builder change is safe to push"	Dry-read the current stored content state; diff against the intended state
   "Deployed config matches the repo"	Fetch the deployed config and diff against the repo file

4. Tabulate findings as Facts with sources (one row per verification command run).

5. Score each sub-claim against the bs-check rubric using the observed evidence. Most Direct-Obs sub-claims will cap at 98%. Apply the standard caps + stability multiplier + lowest-cap-wins rule from Step 5 of the Process section.

6. Tag sub-claim contradictions explicitly. When a documented claim contradicts the live state (the classic doc-drift case), this is a Compare-Obs at very_unlikely / <30% - this drives the BLOCKED gate.

7. Compute the blocking gate. BLOCKED if any Critical sub-claim has a contradiction or is < 60% after caps. Otherwise PASS.

8. Render the standard Full mode output format with the Direct-Obs banner at top declaring the mode substitution.

Hard invariants for Direct-Obs mode

• Modality shift is the safeguard. The research basis identifies modality shift - moving from text reasoning to executable verification against the real system - as the strongest verification pattern, stronger than context isolation. Direct-Obs mode achieves this by definition.
• Every Critical sub-claim must have at least one verification command - no sub-claim is rated highly_probable / near_certain without an actual command run. If you can't reach the system to verify, drop confidence accordingly and tag "verification not possible from current context."
• No silent substitution. The Direct-Obs banner at the top of the output must declare: "Direct-Obs mode used. Stage 1 subagent + Stage 2 web judge bypassed. Justification: [routing reason]. Commands run: [count + key commands]."
• One concern. Direct-Obs mode is judging only - no rewriting configs, no proposing infrastructure fixes, no "let me also restart the container for you." Verification then verdict, nothing else.
• Observability - every verification command is a TodoWrite step.

When Direct-Obs mode cannot complete

State machine:

Full mode pipeline (orchestration)

This section is the orchestration layer for Full mode only. Quick mode skips straight to the Process steps below and runs them inline.

When invoked with Full mode, main-thread Claude MUST walk this checklist top to bottom. Each step is a TodoWrite item so skipped steps are visible in the trace.

Pipeline checklist

1. Create TodoWrite items, one per remaining step (2-7 below).
2. Dispatch Stage 1 subagent. Use the Agent tool with subagent_type: "general-purpose". Pass the Stage 1 prompt template (below). Pass the claim text and bs-check rubric - do NOT pass main-thread reasoning, confidence, or tool-call history. Wait for structured JSON return.
3. Validate Stage 1 output schema. If JSON is malformed -> retry once with stricter prompt. Still malformed -> fall through to Exception handling (below).
4. Filter sub-claims for Stage 2. From the Stage 1 JSON, select sub-claims where:
   • criticality == "critical" AND
   • evidence_type ∈ {"Inference", "Secondhand"} OR confidence_pct < 80 OR the sub-claim matches a fabrication-prone pattern (named entity, citation, statute, date, specific number, pricing tier, model ID, API endpoint).
   • Skip sub-claims where evidence_type ∈ {"Direct-Obs", "Compare-Obs"} with confidence_pct ≥ 80 - no value in asking the web judge to second-guess a verified observation.
5. Call Stage 2 (web judge) per filtered sub-claim. Select the judge by availability:
   • Perplexity MCP connected -> call mcp__perplexity__perplexity_reason with the Stage 2 prompt template (below), search_context_size: "high", strip_thinking: true. Deferred-tool preflight: some harnesses lazy-load MCP tool schemas, so the tool may be absent from your loaded tool list even though the server is connected. If your harness has a schema loader (e.g. ToolSearch with query "select:mcp__perplexity__perplexity_reason" in Claude Code), load the schema before the first call. An input-validation error on a direct call means the schema was not loaded, NOT that the server is down - load and retry before falling back. The response arrives as a JSON object followed by an appended "Citations:" footer - read the JSON, ignore the footer, do not treat the footer as malformed output.
   • Else, built-in web tools available -> dispatch a fresh general-purpose subagent whose prompt is the Stage 2 template plus "Use WebSearch and WebFetch to check the sources." The subagent gets ONLY the sub-claim text and domain tag.
   • Else -> mark all filtered sub-claims support_level: "verifier_failed" and proceed (the exception banner will render).
   • In all cases: pass ONLY the sub-claim text and domain tag. Do NOT pass Stage 1's confidence, rationale, or verdict - Stage 2 must be blind to Stage 1.
6. Apply Stage 3 synthesis rules. Walk the rule table (below) row by row for each sub-claim with a Stage 2 verdict. Compute the blocking gate.
7. Render Full mode output per the format in Step 9 of the Process section, including the Disagreements section and the binary PASS / BLOCKED gate.

Hard invariants

• Context isolation: Stage 1 subagent and the Stage 2 web judge must each be blind to the other's verdict. Independence is the entire reason cross-judge verification beats inline self-audit. Lose it -> lose the value.
• One concern per subagent: Stage 1 ONLY scores. It may not rewrite, suggest fixes, or propose alternatives. Pure judge.
• Structural gate: Full mode output MUST contain an explicit BLOCKED or PASS flag. No soft "consider holding" language at the gate.
• Observability: Every pipeline step writes a TodoWrite. If a step gets skipped, the unchecked todo is the audit trail.

Stage 1 subagent prompt template

When dispatching the Stage 1 subagent (step 2 of the pipeline), use this exact prompt structure. Substitute {CLAIM} with the claim under audit.

You are an independent confidence auditor. You are NOT the author of the claim below. You may not defend, expand, or rewrite the claim. Your only job is to score it against the rubric.

# The claim to audit

{CLAIM}

# Your task

Walk the bs-check rubric (provided below) and produce a strict JSON output. Do not produce free-form prose. Do not produce suggestions for fixing the claim. Score only.

# Rubric (the bs-check process)

1. **Topic-familiarity gate.** Generate 2-3 simple closely-related verifiable questions about the topic and answer them. If you find yourself hedging or generating plausible-sounding answers without specific anchors, hard-cap every factual sub-claim at 60%.

2. **Decompose** the claim into atomic sub-claims. For each, assign:
   - `criticality`: "critical" | "important" | "background"
   - `evidence_type`: "Direct-Obs" | "Compare-Obs" | "Secondhand" | "Inference"
   - `domain`: "public-API" | "general-knowledge" | "internal-system" | "proprietary" | "speculative"
   - `stability`: "High" | "Medium" | "Low" - simulate answering this sub-claim 5 times independently; would they converge on the same answer AND same evidence?

3. **Apply confidence caps** (lowest applicable cap wins, then apply stability multiplier):
   - Direct-Obs firsthand -> max 98% | Compare-Obs -> max 95% | Secondhand -> max 92% | Inference -> max 90%
   - Timing-only causation -> max 70% | Confound not ruled out -> max 70% | Pattern-match only -> max 70%
   - "Likely / probably" without specific evidence -> max 45%
   - Named entity / citation / date / specific number / pricing tier / model ID / API endpoint without quotable source -> max 30%
   - Topic-familiarity gate failed -> max 60% on every factual sub-claim
   - Stability: High × 1.0 | Medium × 0.8 | Low × 0.5 (or hard 60%, whichever is lower)

4. **Pick a confidence bin first, then a number inside it:**
   - very_unlikely (<30) | possible (30-60) | probable (60-80) | highly_probable (80-95) | near_certain (>95, direct observation only)

5. **Set overall verdict:**
   - `overall_verdict`: "pass" if all critical sub-claims ≥ 80%; "fail" if any critical sub-claim < 70%; "needs_more_info" otherwise.
   - `is_blocking_failure`: true if any critical sub-claim is contradicted by evidence in your own decomposition or scored < 60%.

# Required output (strict JSON only, no prose around it)

```json
{
  "sub_claims": [
    {
      "id": 1,
      "text": "<sub-claim text>",
      "criticality": "critical" | "important" | "background",
      "evidence_type": "Direct-Obs" | "Compare-Obs" | "Secondhand" | "Inference",
      "domain": "public-API" | "general-knowledge" | "internal-system" | "proprietary" | "speculative",
      "stability": "High" | "Medium" | "Low",
      "confidence_bin": "very_unlikely" | "possible" | "probable" | "highly_probable" | "near_certain",
      "confidence_pct": 0,
      "rationale": "<1-2 sentences on why this score, what evidence backs it, what would raise it>"
    }
  ],
  "overall_verdict": "pass" | "fail" | "needs_more_info",
  "is_blocking_failure": false
}
```

Return only the JSON. No surrounding markdown, no explanation, no apology.

Stage 2 web-judge prompt template

For each filtered sub-claim (pipeline step 5), send this prompt to the selected web judge (Perplexity MCP, or a fresh web-tool subagent). Substitute {SUB_CLAIM_TEXT} and {DOMAIN_TAG} from the Stage 1 JSON.

The template is built around anti-undersearching patterns from 2024-2026 LLM-as-judge literature: a mandatory multi-source checklist before abstention is allowed, a "show your work" requirement via the sources_checked field, and strict label semantics that exclude marketing-page silence from "unverifiable."

You are auditing a single factual claim against current web sources. Reply ONLY with valid JSON matching the schema below. No prose outside the JSON.

# Claim to audit
"{SUB_CLAIM_TEXT}"

# Domain hint (from upstream)
{DOMAIN_TAG}

# MANDATORY search procedure (do this BEFORE deciding the verdict)

For any vendor-specific claim (a company's API, model offering, endpoint path, pricing tier, version availability, SDK behavior, etc.), you MUST attempt to verify by checking the high-priority sources below before considering "unverifiable":

1. **Vendor model / product catalog** - patterns like `<vendor>.com/models/<slug>`, `/products/<slug>`, `/catalog`, `/marketplace`. Always check the most obvious slug first.
2. **Vendor API or developer docs** - `<vendor>.com/docs`, `/docs/api`, `/api/reference`, `/developers`, and any model-specific sub-path like `/models/<slug>/api`.
3. **Vendor code repositories** - `github.com/<vendor-org>`, official npm packages, PyPI packages, the actual SDK source. Code repositories often contain the exact endpoint paths and version strings that marketing pages omit.
4. **Vendor pricing page** - `<vendor>.com/pricing`, `/plans`, `/billing`, and model-specific pricing pages like `/models/<slug>/pricing`. (For pricing claims.)
5. **Localized docs** - `/docs/<lang>`. Translated docs sometimes contain endpoint tables or version specifics the English pages omit.

Marketing landing pages and SEO blog posts are **LOW-PRIORITY** sources for this audit. Model catalog pages, API docs, code repos, and pricing pages are **HIGH-PRIORITY**.

Also consider reasonable name variants: "Product 2.0" / "Product v2" / "product-2.0".

# Label semantics (strict)

- **supported** - you found clear, specific evidence from at least one HIGH-PRIORITY source (official docs / model page / API docs / code repo / pricing page) that supports the claim.
- **contradicted** - you found HIGH-PRIORITY sources that clearly disagree with the claim.
- **unverifiable** - you have checked the relevant HIGH-PRIORITY sources listed above AND considered name variants, AND still cannot find supporting or contradicting evidence.

**Do NOT use "unverifiable" as a default.** Marketing-page silence is not unverifiable. If you only checked the homepage, blog, or a generic overview, you have not earned the right to abstain - go check the model catalog, API docs, code repo, or pricing page first.

# REQUIRED output (strict JSON, nothing else)

```json
{
  "claim_id": 0,
  "support_level": "supported" | "contradicted" | "unverifiable" | "verifier_failed",
  "domain_match": "well_covered" | "partially_covered" | "niche_or_proprietary",
  "sources_checked": [
    {"location": "<vendor URL or path you checked>", "query_or_path": "<what you searched for>", "result": "FOUND <what> | NO MATCH | PARTIAL MATCH"}
  ],
  "evidence_snippets": [
    {"quote": "<short quote from a HIGH-PRIORITY source>", "source_type": "model_catalog" | "api_docs" | "official_docs" | "pricing_page" | "code_repo" | "community_forum" | "blog" | "news" | "unknown", "source_url": "<url>"}
  ],
  "reasoning": "<1-3 sentences tying evidence to verdict>"
}
```

The `sources_checked` field is **MANDATORY**. If you return "unverifiable" with fewer than 3 HIGH-PRIORITY sources listed in `sources_checked`, your verdict will be rejected by the orchestrator and re-queried with a stricter prompt. List the actual URLs / paths you checked, even if they returned NO MATCH.

Few-shot anchors (for reference, not part of the prompt body)

The literature recommends 2-3 examples that reward deep search. The orchestrator may optionally append these to the prompt for higher-stakes audits:

• The endpoint that marketing pages omit - a vendor's English marketing pages don't quote the API base path, but the official GitHub SDK repo and localized docs do. Correct verdict: supported after checking the code repo and localized docs - not unverifiable after checking the homepage.
• The model page behind the generic homepage - vendor.com is generic, but vendor.com/models/<slug> is a dedicated page with API docs and per-unit pricing. Correct verdict: supported after checking the model catalog directly.
• The headline rate that doesn't generalize - a vendor's advertised price is documented for the lowest tier only; the tier in the claim has its own pricing page. Correct verdict: split into per-tier sub-claims; do not generalize from the headline rate.

Stage 3 synthesis rules

Main thread applies these rules deterministically per sub-claim. Do not freelance - walk the table.

Core principle (research-backed): contradicted is evidence against the claim. unverifiable is abstention - absence of evidence. They must NOT be treated the same in synthesis.

A Stage 2 unverifiable verdict is only meaningful when the web judge has shown its work: the sources_checked field must list ≥3 HIGH-PRIORITY sources actually examined (model catalog / API docs / code repo / pricing page / localized docs). If sources_checked is short, sparse, or weighted toward LOW-PRIORITY sources (marketing, blog, news), Stage 2 has undersearched - re-query before applying any downgrade.

Pre-synthesis: coverage check on unverifiable verdicts

Before walking the rule table, for any sub-claim where Stage 2 returned unverifiable:

1. Count entries in sources_checked where location is a HIGH-PRIORITY source type (model_catalog, api_docs, official_docs, code_repo, pricing_page).
2. If count ≥ 3 -> flag the sub-claim coverage: sufficient and proceed to the table.
3. If count < 3 OR the listed sources are all LOW-PRIORITY -> flag the sub-claim coverage: insufficient and re-query Stage 2 ONCE with the stricter re-query prompt below. After re-query, treat the second verdict as final regardless of its coverage.

Synthesis rule table

Re-query prompt template (for undersearched unverifiable)

When pre-synthesis triggers a re-query, call the web judge again with this stricter prompt. If the judge is Perplexity, also pass search_domain_filter: ["<vendor apex domain>", "github.com"] - use the vendor's actual apex domain as it appears in the claim (do NOT assume .com; subdomains are covered automatically). Sonar models do not honor site: operators written in prompt text - the API parameter is what actually restricts retrieval, so treat the site: lines below as look-for targets, not operators. If the judge is a web-tools subagent, the site: query forms below work as written. Substitute {SUB_CLAIM_TEXT}, {VENDOR} (if a vendor is mentioned), {ENTITY} (the model / endpoint / pricing tier / version being claimed), and {PRIOR_SOURCES_CHECKED} (the JSON from the first attempt's sources_checked).

Your previous audit returned "unverifiable" for the claim below, but `sources_checked` showed insufficient HIGH-PRIORITY source coverage. Re-search with TARGETED queries against the specific high-priority sources you missed.

# Claim
"{SUB_CLAIM_TEXT}"

# Vendor / entity to target
Vendor: {VENDOR}
Entity (model / endpoint / version / pricing tier): {ENTITY}

# Sources you already checked (do not repeat these)
{PRIOR_SOURCES_CHECKED}

# REQUIRED queries this round (try each, list results in sources_checked)

1. site:{VENDOR}.com "{ENTITY}" - direct on-domain search
2. site:{VENDOR}.com/models/{ENTITY-slug} - common model catalog pattern
3. site:{VENDOR}.com/docs OR /docs/api - API documentation
4. site:github.com/{VENDOR}* OR org repos - code / SDK source
5. site:{VENDOR}.com/pricing - pricing tier reference (for pricing claims)
6. Localized docs: site:{VENDOR}.com/docs/<lang> - if English docs are sparse

Reasonable name variants: try at least 2 (e.g., "Product 2.0" / "product-2.0" / "Product v2").

Return the same JSON schema. `sources_checked` MUST list at least 5 NEW source attempts this round. "Unverifiable" is only valid if all 5 came back NO MATCH.

Overall blocking gate

BLOCKED if any Critical sub-claim hits a BLOCKED row above. Otherwise PASS.

If BLOCKED, the recommended action is "Do NOT proceed. Run the validation step in the queue before any further action." No soft caveats at the gate.

Anti-rubber-stamping note

The asymmetric rules above (Direct-Obs trumps unverifiable, LOW-PRIORITY contradiction doesn't auto-BLOCK) are designed to prevent Stage 2 false negatives from punishing true claims. They are NOT a license to ignore Stage 2. Specifically:

• A contradicted verdict from Stage 2 with HIGH-PRIORITY sources cited STILL forces BLOCKED, even if Stage 1 had Direct-Obs.
• The disagreement is surfaced loudly in the Disagreements section regardless of which side "wins" the synthesis.
• The output ALWAYS shows both verdicts side-by-side so the user can override the rule table when their domain knowledge demands it.

Exception handling

State machine for component failures. Each branch ends in a defined fallback so the pipeline never silently degrades.

Hard cap on loops: Maximum 1 retry per stage. Never loop indefinitely. Always exit to a defined fallback state with a visible banner.

Process

This section is the bs-check rubric. It is the shared scoring logic used by:

• Quick mode (inline, in this thread, by main-thread Claude)
• Full mode Stage 1 (dispatched, by the subagent - see Stage 1 prompt template above)

Step 0: Topic-familiarity gate

Before decomposing, run a short in-distribution check on the topic. Generate 2-3 simple, closely related, verifiable questions about the topic and answer them.

• If you can answer cleanly with concrete specifics -> continue normally.
• If you start hedging, generalizing, or producing plausible-sounding answers without specific anchors -> hard-cap every factual sub-claim at 60% and push toward "hold as hypothesis." Note "out-of-distribution: simple-question probe failed."

Example probes by claim type (write your own for your stack - see Domain lenses):

• Workflow-engine claim -> "What's the actual version of this node/step type on the live instance? Does this operation exist on this type? Does the trigger need explicit registration?"
• CMS / page-builder claim -> "What plugin/theme actually controls this element? Is there a global style override in play? Is serialized data involved that a naive write would corrupt?"
• Third-party API claim -> "What's the actual current endpoint, model ID, pricing tier? Have I confirmed this against the live docs recently, or am I going on prior kno

…(truncated)