Mobile Skin Classification Eval

mobile-skin-classification-eval

A Smartphone-Based Skin Disease Classification Using MobileNet CNN — Velasco et al. (2019) (arXiv:1911.07929, 2019)

What this evaluates

Evaluates a CNN model's ability to classify smartphone-captured images of skin lesions into one of seven dermatological conditions. It probes the model's robustness to class imbalance and the effectiveness of data preprocessing strategies like oversampling and augmentation.

Datasets

• Smartphone Skin Disease Dataset — total 3406; splits: train (2724), test (638), validation (638)

Metrics

• accuracy (primary) — range: percent
  • Percentage of correctly classified images out of the total test set. Calculated as (True Positives / Total Samples) * 100.

Input / output format

Input: RGB images of skin lesions captured via smartphone camera.

Output: A single class label from the set: Acne, Eczema, Pityriasis rosea, Psoriasis, Tinea Corporis, Varicella (chickenpox), Vitiligo.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100

Common pitfalls

• The validation and test sets appear to have identical sizes in the tables, suggesting the 20% non-training split may have been used for both validation and testing, or the split ratio is ambiguous.
• Class imbalance is significant (e.g., Acne: 805 vs Varicella: 314), so accuracy alone may mask poor performance on minority classes without per-class metrics.
• Data augmentation parameters are fixed (e.g., rotation 40, shifts 0.2), but the exact implementation details (e.g., TensorFlow/Keras defaults) are not fully specified.

Evidence (verbatim from paper)

Figures 9 to 10 show the confusion matrix of the model with data augmentation and with oversampling technique used. Still, this demonstrate how the system misclassify most of the psoriasis test images as acne and pityriasis rosea but from 75% accuracy on psoriasis, the accuracy goes up to 80% as well as the rank-1 accuracy of the model and is now, 94.4%.

Citation

@misc{velasco2019smartphone,
  title={A Smartphone-Based Skin Disease Classification Using MobileNet CNN},
  author={Velasco et al. (2019)},
  year={2019},
  note={arXiv:1911.07929}
}

• arXiv: 1911.07929