# Deferredseg Eval

> Evaluates a pixel-wise deferral framework for medical image segmentation that dynamically routes uncertain pixels to synthetic or real experts. It measures how well the collaborative system improves segmentation accuracy over strong baselines like MedSAM across diverse organs and imaging modalities. Use when the user wants to benchmark on PROMISE12, LiTS, AMOS22, Chaksu, or asks about evaluating this task. Reports DSC.

- Skill: `qhjqhj00/deferredseg-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/deferredseg-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/deferredseg-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/deferredseg-eval

---


# deferredseg-eval

> DeferredSeg: A Multi-Expert Deferral Framework for Trustworthy Medical Image Segmentation — Tian et al. (2026) (arXiv:2604.12411, 2026)

## What this evaluates

Evaluates a pixel-wise deferral framework for medical image segmentation that dynamically routes uncertain pixels to synthetic or real experts. It measures how well the collaborative system improves segmentation accuracy over strong baselines like MedSAM across diverse organs and imaging modalities.

## Datasets

- **PROMISE12** — total 80; splits: test (-1)
- **LiTS** — total 201; splits: test (-1)
- **AMOS22** — total 600; splits: test (-1)
- **Chaksu** — total 1345; splits: test (-1)

## Metrics

- `DSC` **(primary)** — range: percent
  - Dice Similarity Coefficient: 2|A∩B|/(|A|+|B|), where A and B are the predicted and ground truth masks.
- `Jaccard` — range: percent
  - Intersection over Union: |A∩B|/|A∪B|, measuring overlap between predicted and ground truth masks.
- `Sensitivity` — range: percent
  - True Positive Rate: TP/(TP+FN), measuring the proportion of actual positive pixels correctly identified.

## Input / output format

**Input**: 2D image slices resized to 1024×1024. CT intensities clipped to [-250, 250] HU and normalized to [0,1]; MRI min-max normalized per volume. Training uses random horizontal/vertical flips and 90° rotations.

**Output**: Pixel-wise binary segmentation masks and deferral routing decisions indicating whether each pixel is handled by the model, a synthetic/real expert, or the combined system.

## Scoring recipe

```python
def compute_metrics(pred_mask, gt_mask):
    intersection = np.logical_and(pred_mask, gt_mask).sum()
    union = np.logical_or(pred_mask, gt_mask).sum()
    gt_sum = gt_mask.sum()
    pred_sum = pred_mask.sum()
    dsc = 2 * intersection / (gt_sum + pred_sum + 1e-6)
    jaccard = intersection / (union + 1e-6)
    sensitivity = intersection / (gt_sum + 1e-6)
    return dsc, jaccard, sensitivity
# Report mean ± std across all test volumes/images for each dataset.
```

## Common pitfalls

- Reports three distinct performance perspectives (System, Expert-only, Model-only) that must be carefully distinguished when comparing against baselines.
- Synthetic experts are simulated with fixed accuracy triplets for foreground, background, and boundary regions rather than using real human annotations, except on Chaksu.
- Chaksu ground truth uses a consensus algorithm (STAPLE) over multiple doctor annotations, not individual annotator labels.

## Evidence (verbatim from paper)

> We report DSC, Jaccard, and Sensitivity in all experiments.

## Citation

```bibtex
@misc{tian2026deferredseg,
  title={DeferredSeg: A Multi-Expert Deferral Framework for Trustworthy Medical Image Segmentation},
  author={Tian et al. (2026)},
  year={2026},
  note={arXiv:2604.12411}
}
```

- arXiv: 2604.12411