Bio/Medical Research Epistemics
Domain-specific guardrails for scientific research. Use alongside researcher for the workflow; this skill provides the evidence hierarchy, anti-hallucination rules, and bio-specific failure modes.
Anti-Hallucination Rules (non-negotiable)
Citation requirement: Every non-trivial factual claim needs a resolvable citation (DOI, PMID, ClinicalTrials.gov ID, or official URL). If you can't cite it, label "UNCITED."
No fake citations: Never invent paper titles, authors, journals, or numbers. If you can't find the paper, say so.
Separate evidence layers: Keep strictly distinct:
- (a) In vitro / cell culture
- (b) Animal model (species, dose, route)
- (c) Human observational / GWAS association
- (d) Human RCT — surrogate endpoint (biomarker)
- (e) Human RCT — clinical outcome (patient-important)
- (f) Systematic review / meta-analysis
- (g) Clinical guideline / consensus statement
NEVER let (a-c) substitute for (d-g). Say explicitly: "Mechanistic evidence only; no human clinical trial confirms this."
Quantify uncertainty: Effect sizes need CIs or ranges. State population, comparator, timeframe. For genetic associations: OR + CI + population + MAF.
Genetic claims: Distinguish GWAS association vs functional validation vs clinical actionability. State penetrance. "Associated with" ≠ "causes." Single-SNP interpretation of polygenic traits is usually misleading. PGx claims need CPIC/DPWG level.
Dosing guardrails: Rx = guideline ranges only ("discuss with prescriber"). OTC/supplements = evidence-based ranges if cited. Genotype→dose only with CPIC/DPWG-level evidence, otherwise label INFERENCE.
Evidence Hierarchy
Grade every claim:
| Grade |
Type |
Notes |
| 1 |
Clinical guideline / consensus |
NICE, WHO, AAD, CPIC, DPWG |
| 2 |
Systematic review / meta-analysis |
Cochrane, PRISMA-compliant |
| 3 |
Well-powered RCT |
Pre-registered, independent, adequate N |
| 4 |
Small / pilot RCT |
Underpowered, often industry-funded |
| 5 |
Large observational / cohort |
Adjusted, replicated |
| 6 |
GWAS / genetic association |
Report OR, CI, population, replication |
| 7 |
Animal model |
Species, dose, route — note translatability |
| 8 |
In vitro / cell culture |
Note concentration vs physiological |
| 9 |
Case report / expert opinion |
Lowest weight |
Always note: COI, replication status, sample size, population match, effect size (NNT, ARR, or Cohen's d when available).
Inference Rules
You may reason from first principles, but MUST label it INFERENCE.
Any INFERENCE must include:
- Assumptions stated explicitly
- A minimal derivation (with units)
- Sensitivity: what if the key assumption is 2x off?
Three buckets in every output:
- EVIDENCE — cited, graded
- INFERENCE — derived from evidence + assumptions, labeled
- PRACTICAL — availability, cost, formulation; never upgraded to efficacy claims
Interpretation Compression Rules
Before promoting any new concept into a system, ask:
- Is this a genuinely new primitive, or just a limiter on an existing one?
- Does it have a caller?
- Does it change belief, escalation, contradiction handling, or follow-up ordering?
If the answer is "no" or "mostly wording", do not promote it as a new runtime object. Keep it as memo-level guidance or merge it into an existing operator.
This matters most in genomics and phenotype-policy work, where good epistemic caveats can easily metastasize into a Rube Goldberg system if every caveat becomes a first-class type.
Bio-Specific Failure Modes
Check yourself against each before outputting:
- Genotype→phenotype leap: Treating GWAS association (OR 1.1-1.5) as deterministic prediction. Fix: state OR, CI, population, penetrance.
- Concentration confusion: Citing in vitro effect at 100μM as evidence for 500mg oral supplement without bioavailability discussion. Fix: check if effective concentration is physiologically achievable.
- Supplement industry bias: Most supplement RCTs are small, industry-funded, surrogate endpoints, positive publication bias. Fix: flag funding, N, endpoint type.
- Protocol broadcasting: Treating Huberman/Attia/Sinclair recommendation as evidence. Fix: trace to primary study and grade independently.
- N=1 extrapolation: "Bryan Johnson does X" = anecdote, not evidence.
- False binary: "This SNP means you can't convert X" when actual effect is 20-40% reduction. Fix: quantified ranges, not categorical language.
- Directionality error: Citing real study but inverting the sign. Fix: explicitly state what changes, which moiety, direction for each step.
- Inference promotion: Plausible mechanistic chain presented as decision-grade evidence. Fix: put in explicit INFERENCE section with assumptions + failure modes.
- Genotype-only search: Only searched genotype→supplement, never condition→supplement. Fix: ALWAYS run condition-anchored search axis in parallel.
LLM-Specific Failure Modes (updated Feb 2026)
| Model |
Failure Mode |
Severity |
Notes |
| Claude (Opus 4.6) |
Sycophantic hedging; agrees then qualifies until useless |
Medium |
Improved from 4.5 but still present |
| Claude |
Citation-shaped bullshit; plausible references that don't exist |
High |
CoT unfaithfulness baseline: 7-13% on clean prompts (ICLR 2026) |
| Claude |
Genotype determinism; treats associations as deterministic |
High |
|
| GPT (5.2–5.4) |
Confident fabrication; invents complete fake studies with authors and N |
Critical |
Worse with extended thinking enabled. 5.4 improved (SimpleQA ~72%) but still rarely refuses — fabricates confidently |
| GPT |
Overcitation; cites 20+ papers, many tangential or unverifiable |
Medium |
|
| Gemini (3.1 Pro) |
Google-source bias; over-relies on Scholar snippets without reading papers |
High |
1M context invites dumping entire papers without processing |
| Gemini |
Length inflation; massive outputs that bury the signal |
Medium |
|
| All models |
Implicit post-hoc rationalization; unfaithful CoT on clean prompts |
Medium |
7-13% baseline rate (arXiv, ICLR 2026 submission). Not adversarial — happens on normal prompts |
Cross-model validation: For high-stakes bio claims (Grade 1-3 evidence affecting clinical decisions), route the same evidence through a second model as independent assessor. Different models have different fabrication patterns — Claude invents plausible-but-wrong citations, GPT invents complete fake studies. Cross-checking catches both.
Recitation Before Synthesis
Before grading evidence or writing conclusions, recite the key evidence items verbatim — restate the study name, N, effect size, and population for each Grade 1-5 source you're relying on. This combats lost-in-the-middle effects when working with many sources (Du et al., EMNLP 2025: +4% accuracy, training-free).
Don't summarize — recite. The act of restating forces attention back to the actual data before the synthesis step where hallucination risk is highest.
Self-Audit Checklist
After any bio research output:
PGx Quick Reference
Justified: CPIC Level A/B, PharmGKB Level 1A/1B.
Not justified (but LLMs do it): Single GWAS hit OR<2.0→dose recommendation; nutrigenomic SNP→supplement dose; variant without replication in user's ancestry.
Key databases: CPIC (cpicpgx.org), PharmGKB, ClinVar, DPWG, gnomAD.
Non-Paper Evidence (acceptable, labeled)
- Regulatory: FDA/EMA monographs, drug labels, safety communications
- Grey literature: ClinicalTrials.gov entries, conference posters, preprints [PREPRINT]
- Independent testing: ConsumerLab, Labdoor, third-party CoAs
- PGx databases: PharmGKB, ClinVar, CPIC guidelines
- Operational: formulation stability, supply chain, product CoAs
1---2name: epistemics3description: Bio/medical/scientific evidence hierarchy and anti-hallucination rules. Use when conducting claim-heavy medical research, genomics interpretation, supplement evaluation, pharmacogenomics, or clinical evidence synthesis. NOT for casual health questions, software engineering, or physics. Companion to researcher skill.4user-invocable: false5effort: high6metadata:7 source: https://skillmd.com/skills/markusstrasser/epistemics8---910# Bio/Medical Research Epistemics1112Domain-specific guardrails for scientific research. Use alongside `researcher` for the workflow; this skill provides the evidence hierarchy, anti-hallucination rules, and bio-specific failure modes.1314## Anti-Hallucination Rules (non-negotiable)15161. **Citation requirement:** Every non-trivial factual claim needs a resolvable citation (DOI, PMID, ClinicalTrials.gov ID, or official URL). If you can't cite it, label "UNCITED."17182. **No fake citations:** Never invent paper titles, authors, journals, or numbers. If you can't find the paper, say so.19203. **Separate evidence layers:** Keep strictly distinct:21 - (a) In vitro / cell culture22 - (b) Animal model (species, dose, route)23 - (c) Human observational / GWAS association24 - (d) Human RCT — surrogate endpoint (biomarker)25 - (e) Human RCT — clinical outcome (patient-important)26 - (f) Systematic review / meta-analysis27 - (g) Clinical guideline / consensus statement2829 NEVER let (a-c) substitute for (d-g). Say explicitly: "Mechanistic evidence only; no human clinical trial confirms this."30314. **Quantify uncertainty:** Effect sizes need CIs or ranges. State population, comparator, timeframe. For genetic associations: OR + CI + population + MAF.32335. **Genetic claims:** Distinguish GWAS association vs functional validation vs clinical actionability. State penetrance. "Associated with" ≠ "causes." Single-SNP interpretation of polygenic traits is usually misleading. PGx claims need CPIC/DPWG level.34356. **Dosing guardrails:** Rx = guideline ranges only ("discuss with prescriber"). OTC/supplements = evidence-based ranges if cited. Genotype→dose only with CPIC/DPWG-level evidence, otherwise label INFERENCE.3637## Evidence Hierarchy3839Grade every claim:4041| Grade | Type | Notes |42|-------|------|-------|43| 1 | Clinical guideline / consensus | NICE, WHO, AAD, CPIC, DPWG |44| 2 | Systematic review / meta-analysis | Cochrane, PRISMA-compliant |45| 3 | Well-powered RCT | Pre-registered, independent, adequate N |46| 4 | Small / pilot RCT | Underpowered, often industry-funded |47| 5 | Large observational / cohort | Adjusted, replicated |48| 6 | GWAS / genetic association | Report OR, CI, population, replication |49| 7 | Animal model | Species, dose, route — note translatability |50| 8 | In vitro / cell culture | Note concentration vs physiological |51| 9 | Case report / expert opinion | Lowest weight |5253Always note: COI, replication status, sample size, population match, effect size (NNT, ARR, or Cohen's d when available).5455## Inference Rules5657You may reason from first principles, but MUST label it INFERENCE.5859Any INFERENCE must include:60- Assumptions stated explicitly61- A minimal derivation (with units)62- Sensitivity: what if the key assumption is 2x off?6364Three buckets in every output:651. **EVIDENCE** — cited, graded662. **INFERENCE** — derived from evidence + assumptions, labeled673. **PRACTICAL** — availability, cost, formulation; never upgraded to efficacy claims6869## Interpretation Compression Rules7071Before promoting any new concept into a system, ask:72731. Is this a genuinely new primitive, or just a limiter on an existing one?742. Does it have a caller?753. Does it change belief, escalation, contradiction handling, or follow-up ordering?7677If the answer is "no" or "mostly wording", do not promote it as a new runtime object. Keep it as memo-level guidance or merge it into an existing operator.7879This matters most in genomics and phenotype-policy work, where good epistemic caveats can easily metastasize into a Rube Goldberg system if every caveat becomes a first-class type.8081## Bio-Specific Failure Modes8283Check yourself against each before outputting:8485- **Genotype→phenotype leap:** Treating GWAS association (OR 1.1-1.5) as deterministic prediction. Fix: state OR, CI, population, penetrance.86- **Concentration confusion:** Citing in vitro effect at 100μM as evidence for 500mg oral supplement without bioavailability discussion. Fix: check if effective concentration is physiologically achievable.87- **Supplement industry bias:** Most supplement RCTs are small, industry-funded, surrogate endpoints, positive publication bias. Fix: flag funding, N, endpoint type.88- **Protocol broadcasting:** Treating Huberman/Attia/Sinclair recommendation as evidence. Fix: trace to primary study and grade independently.89- **N=1 extrapolation:** "Bryan Johnson does X" = anecdote, not evidence.90- **False binary:** "This SNP means you can't convert X" when actual effect is 20-40% reduction. Fix: quantified ranges, not categorical language.91- **Directionality error:** Citing real study but inverting the sign. Fix: explicitly state what changes, which moiety, direction for each step.92- **Inference promotion:** Plausible mechanistic chain presented as decision-grade evidence. Fix: put in explicit INFERENCE section with assumptions + failure modes.93- **Genotype-only search:** Only searched genotype→supplement, never condition→supplement. Fix: ALWAYS run condition-anchored search axis in parallel.9495## LLM-Specific Failure Modes (updated Feb 2026)9697| Model | Failure Mode | Severity | Notes |98|-------|-------------|----------|-------|99| Claude (Opus 4.6) | Sycophantic hedging; agrees then qualifies until useless | Medium | Improved from 4.5 but still present |100| Claude | Citation-shaped bullshit; plausible references that don't exist | High | CoT unfaithfulness baseline: 7-13% on clean prompts (ICLR 2026) |101| Claude | Genotype determinism; treats associations as deterministic | High | |102| GPT (5.2–5.4) | Confident fabrication; invents complete fake studies with authors and N | Critical | Worse with extended thinking enabled. 5.4 improved (SimpleQA ~72%) but still rarely refuses — fabricates confidently |103| GPT | Overcitation; cites 20+ papers, many tangential or unverifiable | Medium | |104| Gemini (3.1 Pro) | Google-source bias; over-relies on Scholar snippets without reading papers | High | 1M context invites dumping entire papers without processing |105| Gemini | Length inflation; massive outputs that bury the signal | Medium | |106| All models | Implicit post-hoc rationalization; unfaithful CoT on clean prompts | Medium | 7-13% baseline rate (arXiv, ICLR 2026 submission). Not adversarial — happens on normal prompts |107108**Cross-model validation:** For high-stakes bio claims (Grade 1-3 evidence affecting clinical decisions), route the same evidence through a second model as independent assessor. Different models have different fabrication patterns — Claude invents plausible-but-wrong citations, GPT invents complete fake studies. Cross-checking catches both.109110## Recitation Before Synthesis111112Before grading evidence or writing conclusions, **recite the key evidence items verbatim** — restate the study name, N, effect size, and population for each Grade 1-5 source you're relying on. This combats lost-in-the-middle effects when working with many sources (Du et al., EMNLP 2025: +4% accuracy, training-free).113114Don't summarize — recite. The act of restating forces attention back to the actual data before the synthesis step where hallucination risk is highest.115116## Self-Audit Checklist117118After any bio research output:119- [ ] Every number has a source (DOI/PMID/URL)120- [ ] No study cited that you haven't verified exists121- [ ] In vitro/animal evidence NOT used to justify clinical recommendations122- [ ] Genetic associations include OR, CI, population, penetrance123- [ ] "Cannot/always/never" replaced with quantified ranges124- [ ] Industry-funded studies flagged125- [ ] Supplement doses cite the study they come from126- [ ] Genotype→dosing claims have CPIC/DPWG level or labeled INFERENCE127- [ ] Confidence ratings are honest128- [ ] Counterarguments section exists and is substantive129- [ ] New interpretation ideas passed the caller test130- [ ] New interpretation ideas are not just renamed duplicates of earlier operators131- [ ] Cautions/limiters were not promoted into runtime objects without decision impact132133## PGx Quick Reference134135**Justified:** CPIC Level A/B, PharmGKB Level 1A/1B.136**Not justified (but LLMs do it):** Single GWAS hit OR<2.0→dose recommendation; nutrigenomic SNP→supplement dose; variant without replication in user's ancestry.137138**Key databases:** CPIC (cpicpgx.org), PharmGKB, ClinVar, DPWG, gnomAD.139140## Non-Paper Evidence (acceptable, labeled)141142- Regulatory: FDA/EMA monographs, drug labels, safety communications143- Grey literature: ClinicalTrials.gov entries, conference posters, preprints [PREPRINT]144- Independent testing: ConsumerLab, Labdoor, third-party CoAs145- PGx databases: PharmGKB, ClinVar, CPIC guidelines146- Operational: formulation stability, supply chain, product CoAs