Malimg Eval

malimg-eval

Malware Classification Using Deep Boosted Learning — Muhammad Asam et al. (2021) (arXiv:2107.04008, 2021)

What this evaluates

Evaluates the ability of deep learning models to classify malware families from their visual representations (malware images). It probes feature extraction robustness and classification accuracy on an imbalanced dataset.

Datasets

• MalImg — total ?; splits: train (-1), test (-1)

Metrics

• Accuracy (primary) — range: percent
  • The proportion of correctly classified samples out of the total number of samples.
• F1-Score — range: [0, 1]
  • The harmonic mean of precision and recall, calculated as 2 * (precision * recall) / (precision + recall).
• Precision — range: [0, 1]
  • The ratio of true positive predictions to the total number of positive predictions.
• Recall — range: [0, 1]
  • The ratio of true positive predictions to the total number of actual positives.

Input / output format

Input: Grayscale or RGB images of malware binaries (visual representations).

Output: Discrete malware family/class label.

Scoring recipe

accuracy = sum(pred == gold) / len(gold)
precision = tp / (tp + fp)
recall = tp / (tp + fn)
f1 = 2 * precision * recall / (precision + recall)

Common pitfalls

• Different papers use different train/test splits (e.g., 70-30 vs 60-40), making direct accuracy comparisons misleading.
• The dataset is imbalanced; relying solely on Accuracy can mask poor performance on minority classes, which is why F1-score is recommended as the standard measure.

Evidence (verbatim from paper)

In order to make our proposed malware classification framework more robust, we implemented the architectures in the data distribution of 60-40 ratio. These experiments are performed using (i) softmax probability-based Malware classification, (ii) deep feature and ML-based Malware classification and (iii) our proposed malware classification approach (DFS-MC). These malware classification models are implemented using training from scratch for MalImg data set in the first place and then using pre-trained models for ImageNet. Performance measures like accuracy and F1-score of training from scratch and TL-based implementation are recorded.

Citation

@misc{asam2021malware,
  title={Malware Classification Using Deep Boosted Learning},
  author={Muhammad Asam et al. (2021)},
  year={2021},
  note={arXiv:2107.04008}
}

• arXiv: 2107.04008