Vibe Science

Vibe Science v7.0 TRACE — Observe · Recall · Operate

Research engine: agentic tree search over hypotheses, adversarial review by separate sub-agent, 32 methodology gates (8 schema-enforced in the specification), serendipity detection, hook-based enforcement for the implemented runtime subset, cross-session pattern carry-over, and temporal-decay calibration hints. Infinite loops until discovery.

Runtime boundary note: this skill defines the full methodology. The Claude Code plugin currently hard-enforces only a subset of it (for example DQ4, L-1+, L0, D1, stop-time claim resolution, Salvagente, and TEAM permissions). The rest remains methodology until wired into runtime.

WHY THIS SKILL EXISTS

AI agents in science optimize for completion, not truth. They find strong signals, construct narratives, never search for confounders, and declare "done" prematurely.

Over 21 sprints of real research: the agent would have published a confounded claim (OR=2.30, p < 10^-100 — sign reversed by propensity matching), a physically impossible finding (effect direction contradicted by domain knowledge), a noise signal (Cohen's d = 0.07), and non-generalizable rankings. None were hallucinations — the data was real, the statistics correct. The agent never asked: "What if this is an artifact?"

The solution is not more tools. It is a dispositional change: the system must contain an agent whose ONLY job is to destroy claims.

In Claude Code, R2 is a separate sub-agent launched via the Task tool with its own context window. It never sees the researcher's reasoning or excitement — only claims and evidence. This is native Blind-First Pass by architecture.

The Three Principles

1. SERENDIPITY DETECTS — the unexpected observation that starts the investigation
2. PERSISTENCE FOLLOWS — 5, 10, 20+ cycles of testing, not one-and-done
3. REVIEWER 2 VALIDATES — systematic demolition before publication

Full exposition: references/constitution.md

CONSTITUTION (12 Immutable Laws)

LAW 1: DATA-FIRST — No thesis without evidence from data. NO DATA = NO GO. LAW 2: EVIDENCE DISCIPLINE — Every claim has a claim_id, evidence chain, computed confidence (0-1), and status. LAW 3: GATES BLOCK — 32 quality gates are mandatory checkpoints in the methodology. The implemented runtime subset is mechanical hard-stop enforcement; the remainder stay procedural until wired into code. LAW 4: REVIEWER 2 IS CO-PILOT — R2 can VETO, REDIRECT, FORCE re-investigation. Non-negotiable. LAW 5: SERENDIPITY IS THE MISSION — Hunt for the unexpected at every cycle. Score >= 10 → QUEUE. >= 15 → INTERRUPT. LAW 6: ARTIFACTS OVER PROSE — If a step can produce a file, it MUST. LAW 7: FRESH CONTEXT RESILIENCE — Resumable from STATE.md alone (database enriches but is not required). All context lives in files, never in chat history. LAW 8: EXPLORE BEFORE EXPLOIT — Min 3 draft nodes before promotion. Exploration ratio >= 20%. LAW 9: CONFOUNDER HARNESS — Every quantitative claim: raw → conditioned → matched. Sign change = ARTIFACT. Collapse >50% = CONFOUNDED. Survives = ROBUST. NO HARNESS = NO CLAIM. LAW 10: CRYSTALLIZE OR LOSE — Every result written to file. Context window is a buffer, not memory. LAW 11: LISTEN TO THE USER — When the user corrects direction, follow immediately. No arguing, no continuing on previous path. Three ignored corrections = session failure. LAW 12: INSTINCT — Learned patterns from past sessions inform current behavior. Instincts are weighted suggestions (confidence 0.3-0.9) that decay with time (-0.02/week) and can be overridden by contradicting evidence. An instinct below 0.2 confidence is archived.

Full text + role constraints: references/constitution.md

v6.0 INNOVATIONS (over v5.5)

Retained from v5.5

MULTI-AGENT ARCHITECTURE

R2-DEEP as sub-agent (via Task tool) means it has NO access to the researcher's reasoning. It sees ONLY claims and evidence. This is architecturally superior to same-agent role-play.

Full config: references/multi-agent-config.md · Role definitions: AGENTS.md

SESSION INITIALIZATION

Banner

VIBE SCIENCE v7.0 TRACE — Observe · Recall · Operate
TRACE RUNTIME + REVIEW PROTOCOLS → R2 ENSEMBLE → METHODOLOGY GATES (32 total, runtime subset enforced)
SERENDIPITY RADAR · RESEARCH SPINE · OBSERVER · DQ1-DQ4
PATTERNS · HARNESS HINTS · TEMPORAL DECAY · HANDOFF PROTOCOL
Detect · Persist · Demolish · Discover · Learn

Hook-Based Context Injection

At session start, the SessionStart hook automatically provides:

1. [STATE] — Last session summary (actions, claims created/killed)
2. [ALERTS] — Unresolved observer alerts
3. [R2 CALIBRATION] — Temporal decay hints about R2's historical weaknesses
4. [PATTERNS] — Cross-session learned patterns with confidence scores
5. [PENDING SEEDS] — Serendipity seeds from prior sessions awaiting triage
6. [HARNESS HINTS] — Deterministic carry-over guidance from recurring runtime failures

When the Claude Code plugin is active, this context is injected into the agent's system prompt (~700-850 tokens depending on carry-over). In skill-only mode, this automation layer is not available and context must be reconstructed manually.

If .vibe-science/ exists → RESUME

1. Read STATE.md (includes tree state), last 20 lines of PROGRESS.md
2. Read CLAIM-LEDGER.md frontmatter, SPINE.md last entry
3. Check pending: R2 demands, gate failures, debug nodes, Observer alerts
4. Check injected context: patterns, instincts, R2 calibration hints
5. Resume from "Next Action" in STATE.md
6. Announce: "Resuming RQ-XXX, cycle N, stage S. Tree: X nodes (Y good). Next: [Z]."

If .vibe-science/ does NOT exist → INITIALIZE

1. → Phase 0: SCIENTIFIC BRAINSTORM (mandatory)
2. Gate B0 must PASS before any OTAE cycle
3. Create folder structure, populate STATE.md, PROGRESS.md, SPINE.md

Post-Compaction Recovery

If the context was compacted (auto or manual), the PreCompact hook saved a snapshot to DB:

• Active claims, pending seeds, spine entry count, STATE.md content
• Recovery: SessionStart loads last snapshot → agent has enough context to continue

Full protocol: references/context-resilience.md

PHASE 0: SCIENTIFIC BRAINSTORM (Before Everything)

Not optional. Not skippable.

1. UNDERSTAND — Domain, interests, constraints (ask user, one question at a time)
2. LANDSCAPE — Rapid literature scan (last 3-5 years), field mapping, open debates
3. GAPS — Blue ocean hunting: cross-domain analogies, assumption reversal, scale shifting, contradiction hunting
4. DATA — Reality check: does data exist? Score DATA_AVAILABLE (0-1). LAW 1: NO DATA = NO GO
5. HYPOTHESES — Generate 3-5 testable, falsifiable hypotheses with null hypotheses and predictions
6. TRIAGE — Score: impact x feasibility x novelty x data readiness x serendipity potential (/15)
7. R2 REVIEW — Reviewer 2 challenges direction (BLOCKING: must WEAK_ACCEPT)
8. COMMIT — Lock RQ.md with: question, hypothesis, predictions, success/kill conditions

Gate B0: 3+ gaps with evidence, data confirmed (>= 0.5), falsifiable hypothesis, R2 WEAK_ACCEPT, user approved.

Full protocol: references/brainstorm-engine.md

OTAE-TREE LOOP

OBSERVE → THINK → ACT → EVALUATE → CHECKPOINT → CRYSTALLIZE → loop

Each cycle: ONE meaningful action. Each tree node = one OTAE cycle.

v5.0 FORCED Review Path

Canonical methodology path: SFI injection → BFP Phase 1 (blind) → Full review Phase 2 → V0/J0 judgment → schema / gate consequences.

Runtime note: the plugin currently supports this path mainly through artifact ingestion, downstream calibration, and claim-lifecycle consequences. It does not yet hook-enforce every semantic step of SFI/BFP/V0/J0 end-to-end.

Tree Structure

Tree modes: LINEAR (literature), BRANCHING (experiments), HYBRID (both). Tree search selects next node by confidence + metrics. Each node = one OTAE cycle.

Full protocol: references/loop-otae.md · Tree search: references/tree-search.md

HOOKS ENFORCEMENT

7 hooks enforce selected operational consequences of some laws mechanically. They run as Node.js scripts triggered by Claude Code events; the full methodology remains broader than the currently wired runtime.

All hooks degrade gracefully if the DB is unavailable. They never hard-crash.

Full protocol: references/hook-system.md

CROSS-SESSION LEARNING

Pattern Extraction (at session end)

The Stop hook extracts recurring patterns from cross-session data:

1. GATE_FAILURE_CLUSTER — Same gate failing across 2+ sessions → pattern (e.g., "DQ1 fails when zero-variance columns present")
2. REPEATED_ACTION — Same action+input appearing across 2+ sessions → pattern (e.g., "Same bug fix applied 3 times")
3. CLAIM_LIFECYCLE — Claims killed for same reason across sessions → pattern (e.g., "Confounders kill first quantitative claim every session")

Patterns are stored in the research_patterns DB table with confidence scores. At session start, active patterns are surfaced in the [PATTERNS] context block.

Full protocol: references/pattern-extraction.md

Instinct Model (reference design vs runtime)

The full ECC-style instinct ladder remains the reference design:

• Observation (0.3): Pattern noticed once
• Pattern (0.5): Observed 3+ times
• Instinct (0.7): Confirmed by evidence
• Strong Instinct (0.9): Never contradicted

Current runtime implementation is narrower:

• project-scoped pattern extraction into research_patterns
• SessionStart pattern carry-over via [PATTERNS]
• deterministic advisory carry-over via [HARNESS HINTS]

Global instinct synthesis and automatic archival/strength transitions remain blueprint-level, not fully wired runtime behavior.

Full reference model: references/instinct-model.md

R2 Calibration with Temporal Decay

R2's historical weaknesses are tracked with exponential temporal decay:

weight = exp(-0.02 * ageWeeks)

A weakness from 50 weeks ago contributes only ~37% of its original weight. This prevents stale calibration data from persisting indefinitely.

SessionStart injects calibration hints like: "R2 historically weak on 'batch_effect_check' (decay-weighted score: 2.3). High priority."

Full protocol: references/r2-calibration.md

AGENT HANDOFF PROTOCOL

When transferring work between agents (R2 returning verdict, Explorer reporting branch, stage transitions), use formal handoff documents:

## HANDOFF: [Source Agent] → [Target Agent]
### Context
What was being done, which RQ, which stage, which cycle.
### Findings
Key results, claims affected, metrics.
### Files Modified
File paths with line ranges.
### Open Questions
Unresolved issues requiring attention.
### Recommendations
Suggested next steps.

This prevents context loss during agent transitions and satisfies LAW 7 (resilience) + LAW 10 (crystallize).

Full protocol: references/handoff-protocol.md

5-STAGE EXPERIMENT MANAGER

Full protocol: references/experiment-manager.md

REVIEWER 2 CO-PILOT

4 domain-agnostic reviewers: R2-Methods, R2-Stats, R2-Domain, R2-Engineering.

7 activation modes:

R2 INLINE 7-Point Checklist (v5.5+)

For every finding, before recording in CLAIM-LEDGER:

1. Numbers match source data? (SSOT)
2. Sample size adequate and reported?
3. Alternative explanations considered?
4. Prior art checked? (not rediscovering known result)
5. Confounder risk identified? (even if full harness not yet run)
6. Reproducible? (seed, parameters, data path documented)
7. Terminology consistent across documents?

R2 Behavioral Requirements

• ASSUME every claim is wrong
• SEARCH for prior art, contradictions, artifacts
• DEMAND confounder harness for every quantitative claim (LAW 9)
• REFUSE premature closure — minimum 3 falsification attempts per major claim
• ESCALATE, never soften — each pass MORE demanding
• SALVAGENTE: When killing a claim, R2 MUST produce a serendipity seed
• CALIBRATE (v6.0): Check temporal decay hints from SessionStart context. Prioritize historically weak areas.

Full ensemble protocol: references/reviewer2-ensemble.md

SERENDIPITY RADAR

Three-part process: DETECTION → PERSISTENCE → VALIDATION.

Detection (every EVALUATE): 5 scans — anomalies, cross-branch patterns, contradictions, assumption drift, unexpected metrics.

Response: Score >= 10 → QUEUE. Score >= 15 → INTERRUPT (create serendipity node). Unaddressed flag after 5 cycles → ESCALATED.

Salvagente (v5.0): When R2 kills a claim (INSUFFICIENT/CONFOUNDED/PREMATURE), R2 MUST produce a serendipity seed (schema-validated).

Cross-Session Survival (v6.0): Pending seeds are stored in DB and loaded at session start. Seeds that survive across sessions get priority triage.

Full protocol: references/serendipity-engine.md

GATES (32 Total)

Key Gate Summaries

• G0: Input sanity — data exists, format correct, no corruption
• G1: Schema compliance — data schema matches expectation
• DQ1: Post-extraction — no zero-variance, no leakage, cross-checks match
• DQ2: Post-training — outperforms baseline, no single-feature dominance, stable folds
• DQ3: Post-calibration — plausible range, not suspiciously perfect, adequate sample
• DQ4: Post-finding — numbers match source JSON, sample size reported, alternatives listed
• DD0: Data dictionary — all columns documented before use
• DC0: Design compliance — execution matches research design
• L-1: Literature pre-check — prior art searched before committing direction
• V0: Vigilance metric for SFI-enabled reviews — faults caught (RMS >= 0.80, FAR <= 0.10)
• J0: Judge — R3 meta-review score >= 12/18, no dimension = 0

Full gate definitions: references/gates-complete.md DQ gate protocol: references/dq-gates.md

ENFORCEMENT SCRIPTS

Python scripts for deterministic checks. Exit code 0 = PASS, non-zero = FAIL. They support the methodology and audits, but the authoritative non-bypassable runtime path is the Node hook layer inside Claude Code.

All scripts: Python 3.8+, stdlib only (no external dependencies). Domain-configurable via --config domain-config.yaml.

Script Output Format (all scripts return JSON to stdout)

dq_gate.py — returns {"gate": "DQ1", "status": "PASS"|"FAIL", "checks": [{"check": "zero_variance", "passed": true, "detail": "OK", "flagged": []}]}. Each gate runs 4-5 named checks. Thresholds configurable via --config.

sync_check.py — returns {"status": "PASS"|"FAIL", "total_numbers_in_markdown": 12, "matched": 12, "mismatched": 0, "tolerance": 0.001, "mismatches": [...]}. Skips dates, claim IDs, gate names. Divides percentages by 100.

tree_health.py — returns {"gate": "T3", "status": "PASS"|"FAIL", "checks": [...]}. Checks: good_ratio (>=0.20), exploration_ratio (>=0.20), no_stale_branches (5+ non-improving = stale), branch_diversity (>=2 branches, skipped in LINEAR mode).

gate_check.py — returns {"gate": "B0", "status": "PASS"|"FAIL", "schema_file": "...", "artifact_file": "...", "errors": [...], "error_count": 0}. Lightweight validator (no jsonschema lib).

spine_entry.py — returns {"status": "PASS"|"FAIL", "type": "DATA_LOAD", "action": "...", "entry": "### ..."}. Creates SPINE.md if missing. Use --validate-only to check without writing.

observer.py — returns {"status": "OK"|"WARN"|"HALT", "total_alerts": 0, "halt_count": 0, "warn_count": 0, "info_count": 0, "alerts": [...]}. Exit 1 only on HALT or missing project dir.

FOLDER STRUCTURE

.vibe-science/
├── STATE.md                    # Current state (max 100 lines, rewritten each cycle)
├── PROGRESS.md                 # Append-only log
├── CLAIM-LEDGER.md             # All claims with evidence + confidence
├── SPINE.md                    # Research Spine (structured logbook)
├── ASSUMPTION-REGISTER.md      # All assumptions with risk
├── SERENDIPITY.md              # Unexpected discovery log
├── KNOWLEDGE/                  # Cross-RQ accumulated knowledge
└── RQ-001-[slug]/              # Per Research Question
    ├── RQ.md                   # Question, hypothesis, criteria, kill conditions
    ├── 00-brainstorm/          # Phase 0 outputs
    ├── 01-discovery/           # Literature phase
    ├── 02-analysis/            # Analysis phase
    ├── 03-data/                # Data extraction + validation
    ├── 04-validation/          # Numerical validation
    ├── 05-reviewer2/           # R2 reviews
    ├── 06-runs/                # Run bundles
    ├── 07-audit/               # Decision log + snapshots
    ├── 08-tree/                # Tree search artifacts
    └── 09-writeup/             # Paper drafting

STOP CONDITIONS (checked every cycle)

1. SUCCESS — All criteria satisfied + all findings R2-approved → Stage 5 → Final R2 → EXIT
2. NEGATIVE RESULT — Hypothesis disproven or data unavailable → EXIT with documented negative
3. SERENDIPITY PIVOT — Score >= 15 → triage → create new RQ or queue
4. DIMINISHING RETURNS — cycles > 15 AND new_finding_rate < 1/3 → WARN → 3 targeted cycles or pivot
5. DEAD END — All avenues exhausted → EXIT with what was learned
6. TREE COLLAPSE — T3 fails AND no pending debug → R2 emergency review → pivot or conclude

RESOURCE ROUTING TABLE

Load ONLY when needed. Never load all at once.

DEVIATION RULES