Breast Mass Severity Eval

breast-mass-severity-eval

Predicting the Severity of Breast Masses with Data Mining Methods — Mokhtar et al. (2013) (arXiv:1305.7057, 2013)

What this evaluates

This evaluation probes a model's ability to predict breast cancer severity (benign vs. malignant) using clinical and radiological features. It measures classification performance across multiple metrics to assess diagnostic reliability and clinical utility.

Datasets

• Mammographic mass dataset — total ?; splits: train (-1), test (-1)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Calculated as the ratio of correctly classified cases to the total number of cases. Formula: (TP + TN) / N.
• Sensitivity — range: [0, 1]
  • Refers to the rate of correctly classified positive cases. Formula: TP / (TP + FN). Also known as the True Positive Rate.
• Specificity — range: [0, 1]
  • Refers to the rate of correctly classified negative cases. Formula: TN / (TN + FP).
• Area under the ROC curve — range: [0, 1]
  • Summarizes performance over the whole range of threshold values by plotting sensitivity against one minus specificity. Independent of class prevalence.

Input / output format

Input: Normalized patient age and ordinal/nominal BI-RADS attributes describing mammographic mass characteristics.

Output: Binary class label: 'Benign' or 'Malignant'.

Scoring recipe

def compute_metrics(tp, tn, fp, fn):
    n = tp + tn + fp + fn
    accuracy = (tp + tn) / n
    sensitivity = tp / (tp + fn)
    specificity = tn / (tn + fp)
    return accuracy, sensitivity, specificity

Common pitfalls

• The 70:30 split is fixed and not cross-validated, potentially causing high variance in reported metrics.
• Missing values are imputed using a C&RT model before partitioning, which risks data leakage if the imputation model sees test data.
• Table 2 formatting is inconsistent across models, making it difficult to verify exact test set sizes or confusion matrix totals.

Evidence (verbatim from paper)

Classification accuracy is defined as the ratio of the number of correctly classified cases and is equal to the sum of TP and TN divided by the total number of cases N: Accuracy = (TP + TN) / N. Sensitivity refers to the rate of correctly classified positive and is equal to TP divided by the sum of TP and FN. Sensitivity may be referred as a True Positive Rate. Specificity refers to the rate of correctly classified negative and is equal to the ratio of TN to the sum of TN and FP.

Citation

@misc{mokhtar2013predicting,
  title={Predicting the Severity of Breast Masses with Data Mining Methods},
  author={Mokhtar et al. (2013)},
  year={2013},
  note={arXiv:1305.7057}
}

• arXiv: 1305.7057