Tb Classification Eval

tb-classification-eval

Reliable Tuberculosis Detection using Chest X-ray with Deep Learning, Segmentation and Visualization — Rahman et al. (2020) (arXiv:2007.14895, 2020)

What this evaluates

Evaluates deep learning models' ability to classify chest X-rays as tuberculosis or normal, comparing whole-image vs. lung-segmented inputs.

Datasets

• Kaggle CXR images and lung mask dataset — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Accuracy = (TP + TN) / (TP + FN + FP + TN). Measures the proportion of correctly classified images.
• sensitivity — range: [0, 1]
  • Sensitivity = TP / (TP + FN). Also called recall; measures the proportion of actual TB cases correctly identified.
• specificity — range: [0, 1]
  • Specificity = TN / (FP + TN). Measures the proportion of normal cases correctly identified.
• precision — range: [0, 1]
  • Precision = TP / (TP + FP). Measures the proportion of predicted TB cases that are actually TB.
• AUC — range: [0, 1]
  • Area under the Receiver Operating Characteristic (ROC) curve. Computed by averaging five-fold cross-validation results.
• F1 score — range: [0, 1]
  • F1 = (2 * TP) / (2 * TP + FN + FP). Harmonic mean of precision and recall.

Input / output format

Input: Chest X-ray images (either whole image or lung-segmented mask).

Output: Binary classification label: TB or normal.

Scoring recipe

def compute_metrics(y_true, y_pred):
    TP = sum((y_true == 1) & (y_pred == 1))
    TN = sum((y_true == 0) & (y_pred == 0))
    FP = sum((y_true == 0) & (y_pred == 1))
    FN = sum((y_true == 1) & (y_pred == 0))
    accuracy = (TP + TN) / (TP + FN + FP + TN)
    sensitivity = TP / (TP + FN)
    specificity = TN / (FP + TN)
    precision = TP / (TP + FP)
    f1 = (2 * TP) / (2 * TP + FN + FP)
    return {'accuracy': accuracy, 'sensitivity': sensitivity, 'specificity': specificity, 'precision': precision, 'f1': f1}

Common pitfalls

• The paper uses five-fold cross-validation on a relatively small dataset, which can lead to high variance in metrics.
• Segmentation is performed as a separate preprocessing step; errors in lung mask generation may propagate to classification without being accounted for.
• Dataset size is ambiguously stated (704 images for segmentation vs. 4200 for classification in the brief).

Evidence (verbatim from paper)

The performance of different CNNs for testing dataset was evaluated after the completion of validation phase and was compared using six performance metrics: accuracy, sensitivity or recall, specificity, precision, area under curve (AUC), F1 score.

Citation

@misc{rahman2020reliable,
  title={Reliable Tuberculosis Detection using Chest X-ray with Deep Learning, Segmentation and Visualization},
  author={Rahman et al. (2020)},
  year={2020},
  note={arXiv:2007.14895}
}

• arXiv: 2007.14895