Retinal Vessel Segmentation Eval

retinal-vessel-segmentation-eval

Full-scale Representation Guided Network for Retinal Vessel Segmentation — Seo et al. (2025) (arXiv:2501.18921, 2025)

What this evaluates

Evaluates a model's ability to segment retinal blood vessels in fundus images, focusing on preserving fine, elongated vascular structures and handling varying image resolutions and pathologies. The protocol tests robustness under strict hyperparameter consistency and standardized data splits.

Datasets

• DRIVE — total 40; splits: train (20), val (20)
• STARE — total 20; splits: train (10), val (10)
• CHASE_DB1 — total 28; splits: train (14), val (14)
• HRF — total 45; splits: train (22), val (23)

Metrics

• F1 score (primary) — range: [0, 1]
  • Harmonic mean of precision and recall: F1 = 2 * (precision * recall) / (precision + recall). Computed on binary segmentation masks after applying a 0.5 threshold to the model's output probability map.

Input / output format

Input: Single retinal fundus image, resized/padded to dataset-specific square dimensions during training/inference (DRIVE: 608×608, STARE: 704×704, CHASE_DB1: 1024×1024, HRF: 1344×1344). Training uses random crops of 288×288 with augmentations (blur, jitter, flip, perspective, resize, CutMix).

Output: Binary segmentation mask of the same spatial dimensions as the input, where pixels ≥ 0.5 are classified as vessels and < 0.5 as background.

Scoring recipe

def compute_f1(preds, gold, threshold=0.5):
    preds_bin = (preds >= threshold).astype(int)
    tp = np.sum(preds_bin & gold)
    fp = np.sum(preds_bin & ~gold)
    fn = np.sum(~preds_bin & gold)
    precision = tp / (tp + fp + 1e-7)
    recall = tp / (tp + fn + 1e-7)
    return 2 * precision * recall / (precision + recall + 1e-7)

Common pitfalls

• Using different expert annotations (e.g., Hoover B instead of Hoover A for STARE, or 2ndHO for CHASE_DB1) breaks comparability and inflates/deflates scores.
• Varying data augmentation or hyperparameters significantly impacts results due to small dataset sizes; strict consistency is required for fair comparison.
• Inconsistent padding/resizing before inference alters effective resolution and degrades segmentation of fine vessels, lowering F1 scores.

Evidence (verbatim from paper)

To measure the performance of the models, it is necessary to divide the data into training and validation sets. As the retinal vessel segmentation dataset was relatively limited, we split the data into a 1:1 ratio of the training and validation sets. The DRIVE dataset was officially divided into training and validation sets, each containing 20 images. For the STARE, CHASE_DB1, and HRF datasets, we used the first half as training and the remaining half as validation. To select the optimized model during the training step, we used the highest F1 score [44], with an early stop of 400 epochs.

Citation

@misc{seo2025fsgnet,
  title={Full-scale Representation Guided Network for Retinal Vessel Segmentation},
  author={Seo et al. (2025)},
  year={2025},
  note={arXiv:2501.18921}
}

• arXiv: 2501.18921