# Fed Echo Eval

> Evaluates federated learning algorithms on echocardiogram video segmentation tasks under label incompleteness and high data heterogeneity across multiple medical institutions. It probes how FL methods handle missing annotations and conflicting labels from different clinical sites. Use when the user wants to benchmark on Fed-ECHO, or asks about evaluating this task. Reports DICE.

- Skill: `qhjqhj00/fed-echo-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/fed-echo-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/fed-echo-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/fed-echo-eval

---


# fed-echo-eval

> FedCVD: The First Real-World Federated Learning Benchmark on Cardiovascular Disease Data — Zhang et al. (2024) (arXiv:2411.07050, 2024)

## What this evaluates

Evaluates federated learning algorithms on echocardiogram video segmentation tasks under label incompleteness and high data heterogeneity across multiple medical institutions. It probes how FL methods handle missing annotations and conflicting labels from different clinical sites.

## Datasets

- **Fed-ECHO** — total ?; splits: train (-1), test (-1); repo https://github.com/SMILELab-FL/FedCVD

## Metrics

- `DICE` **(primary)** — range: percent
  - Dice similarity coefficient measuring overlap between predicted and ground truth segmentation masks, expressed as a percentage.
- `Hausdorff distance ($d_H$)` — range: other
  - Maximum distance between the boundaries of the predicted and ground truth segmentation masks.

## Input / output format

**Input**: Echocardiogram video frames from a specific medical institution (client).

**Output**: Segmentation masks for cardiac regions (BG, LV_Endo, LV_Epi, LA).

## Scoring recipe

```python
def compute_dice(pred_mask, gt_mask):
    intersection = np.sum(pred_mask & gt_mask)
    union = np.sum(pred_mask | gt_mask)
    return 2 * intersection / union * 100 if union > 0 else 0.0
def compute_hd(pred_mask, gt_mask):
    pred_pts = np.argwhere(pred_mask)
    gt_pts = np.argwhere(gt_mask)
    if len(pred_pts) == 0 or len(gt_pts) == 0: return float('inf')
    dists = cdist(pred_pts, gt_pts)
    return np.max(np.min(dists, axis=1)) + np.max(np.min(dists, axis=0))
```

## Common pitfalls

- Label incompleteness means some institutions only annotate specific regions; 'Maybe-BG' labels from other institutions must be masked during loss calculation and evaluation.
- Evaluating on global test set without masking unreliable labels inflates error metrics.
- Semi-supervised baselines require extra rounds for labeled client training before FL begins.

## Evidence (verbatim from paper)

> Table 4: The performance of different FL methods on Fed-ECHO, with DICE (%) and $d_{H}$ representing DICE index and Hausdorff distance respectively.

## Citation

```bibtex
@misc{zhang2024fedcvd,
  title={FedCVD: The First Real-World Federated Learning Benchmark on Cardiovascular Disease Data},
  author={Zhang et al. (2024)},
  year={2024},
  note={arXiv:2411.07050}
}
```

- arXiv: 2411.07050