Mammofl Eval

mammofl-eval

MammoFL: Mammographic Breast Density Estimation using Federated Learning — Muthukrishnan et al. (arXiv:2206.05575, 2022)

What this evaluates

Evaluates a federated learning framework for quantitative breast density estimation from mammographic images. It probes the model's ability to segment breast and dense tissue, predict percent density, and generalize across different medical institutions while preserving patient privacy.

Datasets

• MC — total ?; splits: test (-1)
• UPHS — total ?; splits: test (-1)

Metrics

• PD MAE (primary) — range: percent
  • Mean Absolute Error between predicted and ground-truth percent density values. Calculated as the average of absolute differences across all test instances.
• PD correlation (Spearman ρ) — range: [-1, 1]
  • Spearman rank correlation coefficient between predicted and ground-truth percent density values, measuring monotonic relationship strength.
• Segmentation DSC — range: [0, 1]
  • Dice Similarity Coefficient for breast and dense tissue masks. Calculated as 2 * |intersection| / (|mask_pred| + |mask_true|).

Input / output format

Input: Mammographic images from MC and UPHS institutions.

Output: Predicted percent density (PD) value; binary segmentation masks for breast and dense tissue.

Scoring recipe

def compute_metrics(pred_pd, gold_pd, pred_mask, gold_mask):
    mae = np.mean(np.abs(pred_pd - gold_pd))
    rho, _ = spearmanr(pred_pd, gold_pd)
    intersection = np.logical_and(pred_mask, gold_mask).sum()
    union = pred_mask.sum() + gold_mask.sum()
    dsc = 2 * intersection / union if union > 0 else 0.0
    return {'PD MAE': mae, 'PD correlation': rho, 'DSC': dsc}

Common pitfalls

• LIBRA-generated PD labels are synthetic and only moderately correlated with gold-standard Cumulus labels (ρ=0.7010); evaluating solely against LIBRA overestimates clinical performance.
• Single-institution models show high metrics on same-institution test data but fail to generalize to other institutions, which can mislead if cross-institution robustness is the goal.
• Statistical significance (p < 0.05) is required to claim outperformance; FL consistently underperforms centralized baselines but differences may be small.

Evidence (verbatim from paper)

The model trained on both datasets with centralized learning resulted in PD MAEs of 3.7206 ± 4.2591 and 3.4971 ± 4.0069, PD correlations of 0.7977 and 0.7893, breast segmentation DSCs of 0.9902 ± 0.0203 and 0.9722 ± 0.0287, and dense tissue segmentation DSCs of 0.7665 ± 0.1465 and 0.6846 ± 0.2021, on the MC and UPHS holdout test datasets, respectively.

Citation

@misc{muthukrishnan2022mammofl,
  title={MammoFL: Mammographic Breast Density Estimation using Federated Learning},
  author={Muthukrishnan et al.},
  year={2022},
  note={arXiv:2206.05575}
}

• arXiv: 2206.05575