# Hest Benchmark Eval

> Evaluates the ability of histopathology foundation models to predict gene expression levels from H&E-stained whole-slide image patches. It probes the alignment between morphological features and transcriptomic profiles across diverse cancer types and organs. Use when the user wants to benchmark on HEST-Benchmark, or asks about evaluating this task. Reports Pearson correlation.

- Skill: `qhjqhj00/hest-benchmark-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/hest-benchmark-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/hest-benchmark-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/hest-benchmark-eval

---


# hest-benchmark-eval

> HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis — Jaume et al. (2024) (arXiv:2406.16192, 2024)

## What this evaluates

Evaluates the ability of histopathology foundation models to predict gene expression levels from H&E-stained whole-slide image patches. It probes the alignment between morphological features and transcriptomic profiles across diverse cancer types and organs.

## Datasets

- **HEST-Benchmark** — total ?; splits: train (-1), test (-1); repo https://github.com/mahmoodlab/hest

## Metrics

- `Pearson correlation` **(primary)** — range: [-1, 1]
  - Computes the Pearson correlation coefficient between predicted and measured log1p-normalized gene expression values for the top 50 highly variable genes. The final reported score is the mean correlation across all genes and cross-validation folds.

## Input / output format

**Input**: 112×112 μm H&E-stained image patches (224×224 pixels at 20× magnification) from whole-slide images.

**Output**: Predicted log1p-normalized expression values for the top 50 highly variable genes per patch.

## Scoring recipe

```python
def score(predictions, gold):
    # predictions, gold: arrays of shape (n_samples, 50)
    # Compute Pearson correlation for each gene
    corrs = [pearsonr(gold[:, i], predictions[:, i])[0] for i in range(50)]
    return np.mean(corrs)
```

## Common pitfalls

- Avoid patient-level data leakage by using patient-stratified splits instead of random patch-level splits.
- Directly applying Ridge regression to models with varying embedding dimensions unfairly penalizes larger embeddings; PCA reduction to 256 factors is recommended for fair comparison.
- Performance varies significantly across cancer types, with some tasks (e.g., READ, ccRCC) showing inherently low correlation due to biological or technical noise.

## Evidence (verbatim from paper)

> All tasks are evaluated using the Pearson correlation between the predicted and measured gene expression. We report mean and standard deviation across all folds (or patients). To avoid train/test patient-level data leakage, we use patient-stratified splits, resulting in a k-fold cross-validation, where k is the number of patients.

## Citation

```bibtex
@misc{jaume2024hest1k,
  title={HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis},
  author={Jaume et al. (2024)},
  year={2024},
  note={arXiv:2406.16192}
}
```

- arXiv: 2406.16192