aAAaqwq 🦖 Semantic Similarity Index
Measure how isolated or connected disease research is across the global biomedical literature, using PubMedBERT embeddings on PubMed abstracts spanning 175 GBD diseases.
What it does
- Takes a disease list (GBD taxonomy) as input
- Retrieves PubMed abstracts (2000-2025) for each disease with quality filtering
- Generates 768-dimensional PubMedBERT embeddings for every abstract
- Computes four semantic equity metrics per disease:
- Semantic Isolation Index (SII): average cosine distance to k-nearest disease neighbours; higher = more isolated, less connected research
- Knowledge Transfer Potential (KTP): cross-disease centroid similarity; higher = more potential for research spillover
- Research Clustering Coefficient (RCC): within-disease embedding variance; higher = more diverse research approaches
- Temporal Semantic Drift: cosine distance between yearly centroids; measures how research focus evolves
- Generates publication-quality multi-panel figures:
- Panel A: Semantic isolation by disease category (boxplot)
- Panel B: Top 20 most semantically isolated diseases (bar chart, NTD/Global South colour-coded)
- Panel C: Semantic isolation vs research volume (scatter with regression)
- Panel D: NTD vs non-NTD significance test (Welch's t-test, Cohen's d)
- Produces a markdown report with all metrics, rankings, and reproducibility bundle
Why this exists
If you ask ChatGPT to "measure research neglect for diseases," it will:
- Not know which embedding model to use for biomedical text
- Hallucinate metrics that sound plausible but have no methodological grounding
- Skip quality filtering (year coverage, abstract coverage, minimum papers)
- Not handle MPS acceleration or checkpointed batch processing
- Produce a single scatter plot with no disease classification
This skill encodes the correct methodological decisions:
- Uses PubMedBERT (the gold-standard biomedical language model)
- Fetches from PubMed with exponential backoff and NCBI rate limiting
- Quality filters: year coverage >= 70%, abstract coverage >= 95%, minimum 50 papers
- Batch embedding with Apple MPS acceleration and CPU fallback
- Checkpointed processing (resume after interruption)
- HDF5 storage with gzip compression and SHA-256 checksums
- Classification against WHO NTD list and Global South priority diseases
- Statistical significance testing (Welch's t-test, Cohen's d)
Key Finding
Neglected tropical diseases (NTDs) are significantly more semantically isolated than other conditions (P < 0.001, Cohen's d = 0.8+). They exist in knowledge silos with limited cross-disciplinary research bridges. The 25 most isolated diseases are disproportionately Global South priority conditions.
Pipeline
05-00-heim-sem-setup.py # Validate environment, create directories
05-01-heim-sem-fetch.py # Retrieve PubMed abstracts (checkpointed)
05-02-heim-sem-embed.py # Generate PubMedBERT embeddings (MPS/CPU)
05-03-heim-sem-compute.py # Compute SII, KTP, RCC, temporal drift
05-04-heim-sem-figures.py # Generate publication figures
05-05-heim-sem-integrate.py # Merge with biobank + clinical trial dimensions
Demo (works out of the box)
python semantic_sim.py --demo --output demo_report
The demo uses pre-computed embeddings and metrics for 175 GBD diseases and generates the full 4-panel figure instantly.
Example Output
Semantic Similarity Index
=========================
Diseases analysed: 175
Total PubMed abstracts: 13,100,000
Embedding model: PubMedBERT (768-dim)
Metric Ranges:
SII: 0.0412 - 0.1893
KTP: 0.6234 - 0.9187
RCC: 0.0891 - 0.3421
Key Finding:
NTDs show +38% higher semantic isolation
P < 0.0001, Cohen's d = 0.84
14/25 most isolated diseases are Global South priority
Figures saved to: demo_report/
Fig5_Semantic_Structure.png (300 dpi)
Fig5_Semantic_Structure.pdf (vector)
Reproducibility:
commands.sh | environment.yml | checksums.sha256
Interpretation Guide
- High SII: Disease research exists in a knowledge silo; limited cross-disciplinary bridges
- Low KTP: Research on this disease has few methodological overlaps with others
- High RCC: Diverse research approaches within the disease (many subtopics)
- High Temporal Drift: Research focus has shifted significantly over time
- NTDs shown in red, Global South diseases in orange, others in grey
- The scatter plot (Panel C) reveals the inverse relationship between research volume and isolation
Citation
If you use this skill in a publication, please cite:
- Corpas, M. et al. (2026). HEIM: Health Equity Index for Measuring structural bias in biomedical research. Under review.
- Corpas, M. (2026). ClawBio. https://github.com/ClawBio/ClawBio