Content Quality Audit

Content Quality Audit

Score any article against the sirsadalot quality bar. Run before publishing — catches the gaps that separate "good enough" from "someone would pay for this."

Scoring Rubric (0-100)

Dimension 1: Citation Density (0-25 points)

Checklist:

• Every mechanism claim has a citation
• Every statistic has a source with year
• Citations use Author et al. (YEAR), Journal format
• At least one citation from the last 2 years
• At least one contrarian/disconfirming paper cited

Dimension 2: Mechanism Depth (0-25 points)

Checklist:

• Molecular steps are explained in order
• Each step has a citation
• Competing mechanism hypotheses mentioned
• Diagram or visual accompanies the mechanism section
• Reader can trace the pathway without prior knowledge

Dimension 3: AI-Tell Score (0-20 points)

Loaded from humanizer skill patterns. Score is inverse — higher is better:

Quick scan checklist:

• No "in today's rapidly evolving landscape"
• No "plays a crucial role in"
• No "further research is needed"
• No "-ing phrase, main clause" tack-on pattern
• Sentence length varies (2-word to 35-word range)
• At least one opinionated statement
• At least one "I don't know" or uncertainty admission
• No bolded key terms (if used, flagged)

Dimension 4: Information Density (0-15 points)

Per-paragraph audit. Each paragraph must contain at least one of:

• A specific number with units
• A citation with journal and year
• A mechanism name (receptor, kinase, pathway)
• A comparative statistic
• A dollar amount or cost figure
• A named risk or diagnostic criterion

Dimension 5: Visual Completeness (0-10 points)

Visual types: comparison chart, mechanism diagram, protocol table, dose-response curve, forest plot, infographic, concept diagram, excalidraw map.

Dimension 6: Structural Completeness (0-5 points)

sirsadalot structure check:

• Bottom Line box or TLDR at the top (1 pt)
• Context/why-this-matters section (1 pt)
• Mechanism deep-dive section (1 pt)
• Practical protocols/dosing section (1 pt)
• Further Reading with annotated papers (1 pt)

How to Run the Audit

# Point it at your article
python3 << 'PYEOF'
article_path = '/home/lenovo/writing/ketamine-mechanisms-2026/final.md'
with open(article_path) as f:
    text = f.read()

# Count citations
import re
citations = re.findall(r'\((\d{4})\)', text)  # years in parens
journals = re.findall(r'\*(Biological|Nature|Science|Neuron|Journal|Archives|American)[^*]*\*', text)
print(f"Unique citation years: {len(set(citations))}")
print(f"Journal mentions: {len(journals)}")

# Count paragraphs with concrete elements
paragraphs = [p for p in text.split('\n\n') if len(p) > 100]
has_number = sum(1 for p in paragraphs if re.search(r'\d+%|\d+ mg|\d+\.\d+', p))
has_citation = sum(1 for p in paragraphs if re.search(r'\(\d{4}\)', p))
print(f"Paragraphs: {len(paragraphs)}")
print(f"With numbers: {has_number} ({100*has_number//max(len(paragraphs),1)}%)")
print(f"With citations: {has_citation} ({100*has_citation//max(len(paragraphs),1)}%)")

# Count visuals referenced
visuals = re.findall(r'!\[.*?\]\(.*?\)|chart|diagram|figure|infographic|table', text, re.IGNORECASE)
print(f"Visual references: {len(visuals)}")

# AI tell scan
ai_tells = ['crucial role', 'rapidly evolving', 'in today's', 'further research is needed',
            'plays a key role', 'it is important to note', 'has been shown to']
for tell in ai_tells:
    if tell.lower() in text.lower():
        print(f"AI TELL FOUND: '{tell}'")

# Structure check
structure_checks = {
    'Bottom Line / TLDR': any(x in text[:500].lower() for x in ['bottom line', 'tldr', 'verdict', 'takeaway']),
    'Context section': 'context' in text.lower() or 'why this matters' in text.lower(),
    'Mechanism deep-dive': 'mechanism' in text.lower() and 'pathway' in text.lower(),
    'Protocol section': 'protocol' in text.lower() or 'dose' in text.lower() or 'dosing' in text.lower(),
    'Further Reading': 'further reading' in text.lower() or 'references' in text.lower(),
}
for check, passed in structure_checks.items():
    print(f"{'[✓]' if passed else '[ ]'} {check}")
PYEOF

Scorecard Output Template

╔══════════════════════════════════════╗
║     CONTENT QUALITY AUDIT           ║
║     Article: [title]                ║
╠══════════════════════════════════════╣
║                                      ║
║  Citation Density:     XX/25  [bar]  ║
║  Mechanism Depth:      XX/25  [bar]  ║
║  AI-Tell Score:        XX/20  [bar]  ║
║  Information Density:  XX/15  [bar]  ║
║  Visual Completeness:  XX/10  [bar]  ║
║  Structure:            XX/5   [bar]  ║
║                            ─────     ║
║  TOTAL:                XX/100        ║
║                                      ║
║  sirsadalot threshold: 85+           ║
║  Publishable:          70+           ║
║  Needs work:           <70           ║
╚══════════════════════════════════════╝

FIXES REQUIRED:
1. [Most impactful gap]
2. [Second]
3. [Third]

PITFALLS

1. Don't score before the humanization pass. Running the audit on a first draft will give artificially low AI-tell scores. Audit the final version.
2. Citation counting is mechanical but meaningful. Regex can't tell if a citation is real or hallucinated. Spot-check 3 citations manually.
3. Information density varies by section. A "Further Reading" section will naturally have lower density than the mechanism section. Weight the mechanism and evidence sections more heavily.