Evidence First Claims

Evidence-first claims

Say in one short line that you are using the evidence-first-claims skill before you act, so the user can see which discipline you are working under.

A claim is only as strong as the lineage attached to it. The engine enforces this in code, so an unsupported claim raises ValueError rather than quietly passing.

The CLI has no command for evidence or claims. Use the MCP tools add_source, add_evidence, register_hypothesis, add_assumption and register_claim. They validate arguments and return contract failures as readable errors. The Python API below is the fallback when the tools are unavailable or you need something they do not expose.

Run the Python API with the interpreter that has scientistos installed, which is the project venv (.venv/bin/python). A system python3 will not have it importable. ResearchEngine(Path("research.db")) resolves relative to the current directory, so cd into the project that owns the graph, or pass an absolute path. Do not create a new graph in whatever directory you happen to be in.

The evidence ladder

Enforced in ResearchEngine._validate_claim_evidence (src/scientistos/engine.py:191):

Two hard rules on top:

• A causal claim (ClaimType.CAUSAL) requires E4 or higher. You cannot say X causes Y off a literature reading and one run.
• E1 and above require the evidence's four verification flags to all be true: existence, locator, entailment, scope_match. Setting them is an assertion that you personally checked the passage. Do not set them to pass a gate.

Config defaults in config/example.yaml: abstract minimum E3, strong conclusion minimum E3, causal minimum E4.

Recording evidence and a claim

from pathlib import Path
from scientistos.engine import ResearchEngine
from scientistos.models import EvidenceLevel, EvidenceVerification, SourceRecord, ClaimType

engine = ResearchEngine(Path("research.db"))

src = engine.add_source(SourceRecord(title="Smith 2024, tropospheric delay", doi="10.1234/x", year=2024))

ev = engine.add_evidence(
    src.id,
    locator="section 4.2, p. 11",        # must point at the exact place, not the whole paper
    excerpt="...verbatim text you actually read...",
    summary="Reports a 3 mm delay bias under high aerosol optical depth.",
    relevance=0.9,
    support=0.8,                          # negative value records contradicting evidence
    verification=EvidenceVerification(
        existence=True, locator=True, entailment=True, scope_match=True,
        checked_by="claude-code-read-passage",
    ),
)

claim = engine.register_claim(
    "High aerosol optical depth is associated with a positive VLBI group delay bias",
    level=EvidenceLevel.E1,
    confidence=0.7,
    claim_type=ClaimType.DESCRIPTIVE,
    evidence_ids=[ev.id],
    qualifiers=["single site", "one observing season"],
)
print(claim.id)

Other engine methods: register_hypothesis, add_assumption, register_experiment, approve, execute_experiment, record_result, record_reproduction, record_audit, register_figure, impact.

Choosing the level

Pick the level the lineage supports, then stop. If register_claim raises, that is the system working. The fix is to get the missing evidence, not to change the level you pass in or weaken the verification flags.

Common honest outcomes:

• Read three papers, ran nothing: E1, descriptive, with qualifiers.
• Ran your own analysis once with a confidence interval: E3 at best.
• Want to say "causes": you need E4, so you need an independent reproduction of every supporting result. Until then, say "is associated with".

Counter-evidence

Record contradicting evidence with negative support. The engine wires it as an opposes edge automatically. Never drop evidence that cuts against your hypothesis. A claim with opposing evidence and no qualifiers raises a warning in the integrity audit, so state the scope limit explicitly.

The two roles

Work the two roles separately, and do not let one soften the other.

As Scientist: propose alternatives, predictions and discriminating experiments. Mark assumptions explicitly with add_assumption. Prefer an executable test to an argument.

As Auditor: try to falsify, not to improve. Run science falsification-plan for the standard attack list, and work through confounding, selection, leakage, measurement, misspecification, multiplicity, coding defects, identifiability, and synthetic-null behaviour. For an important computational result, demand an independent replay before accepting it. Record what you find with record_audit rather than only saying it in chat.

The design assumes the Scientist and the Auditor are different model families (ModelRouter.independent_judges, src/scientistos/providers/router.py:38). When you run both roles yourself, that independence is weaker than the design intends. Say so when it matters, and prefer an external reproduction for anything load-bearing.

Before you write prose

Any sentence in an abstract, results section or summary should map to a claim in the graph at the required level. If you are writing a conclusion that has no claim behind it, either register the claim properly or mark the gap [MISSING EVIDENCE]. Do not strengthen causal language during writing.