Qml Malware Classification Eval

qml-malware-classification-eval

Towards Quantum Machine Learning for Malicious Code Analysis — Lopez et al. (2025) (arXiv:2508.19381, 2025)

What this evaluates

This benchmark evaluates hybrid quantum-classical neural networks (QMLP and QCNN) on malware classification tasks. It probes the ability of NISQ-era quantum circuits to encode high-dimensional static feature vectors and learn discriminative patterns for binary and multiclass malicious software detection.

Datasets

• API-Graph — total 137586; splits: train (79101), test (58485)
• EMBER-Domain — total 216823; splits: train (141817), test (75006)
• AZ-Domain — total 135986; splits: train (67656), test (68330)
• EMBER-Class — total 244857; splits: train (220412), test (24445)
• AZ-Class — total 204824; splits: train (184342), test (20482)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Fraction of correctly predicted samples over the total number of samples. Calculated as TP / (TP + FP + TN + FN).
• Precision — range: [0, 1]
  • Ratio of true positive predictions to the total number of positive predictions. Calculated as TP / (TP + FP).
• Recall — range: [0, 1]
  • Ratio of true positive predictions to the total number of actual positives. Calculated as TP / (TP + FN).
• F1 Score — range: [0, 1]
  • Harmonic mean of Precision and Recall. Calculated as 2 * (Precision * Recall) / (Precision + Recall).
• FPR — range: [0, 1]
  • False Positive Rate. Ratio of false positives to the total number of actual negatives. Calculated as FP / (FP + TN).
• FNR — range: [0, 1]
  • False Negative Rate. Ratio of false negatives to the total number of actual positives. Calculated as FN / (FN + TP).
• ROC-AUC — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve. Measures the model's ability to distinguish between classes across all classification thresholds.

Input / output format

Input: 16-dimensional static malware feature vectors (e.g., API calls, domain features) normalized via Min-Max scaling and reduced via PCA to 16 dimensions, then angle-embedded into a 16-qubit quantum state.

Output: Predicted malware class label (binary: malicious/benign; multiclass: specific family among 4, 14, or 23 families).

Scoring recipe

def compute_metrics(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    accuracy = correct / len(gold_labels)
    # Precision, Recall, F1, FPR, FNR, ROC-AUC computed via standard confusion matrix / sklearn
    return {'accuracy': accuracy}

Common pitfalls

• Evaluation uses ideal (noise-free) qubit simulation rather than real quantum hardware, ignoring decoherence and gate errors.
• Binary datasets use a temporal split (train on past years, test on future years) rather than random shuffling, which may not reflect cross-sectional generalization.
• PCA reduces high-dimensional features to exactly 16 dimensions to match qubit count, potentially discarding discriminative information.

Evidence (verbatim from paper)

TABLE I: Summary of Results. Multi class classification with Quantum Multilayer Perceptron (QMLP) and Quantum Convolutional Neural Network (QCNN). ... For binary classification with AZ-Domain dataset, we use training data from 2008 to 2010, while evaluation is performed on samples from 2011 to 2016. ... Both quantum models are simulated using PennyLane’s default.qubit backend, which provides ideal (noise-free) quantum outputs. ... The final results are reported as the average across these independent runs to capture model consistency and variability.

Citation

@misc{lopez2025quantummalware,
  title={Towards Quantum Machine Learning for Malicious Code Analysis},
  author={Lopez et al. (2025)},
  year={2025},
  note={arXiv:2508.19381}
}

• arXiv: 2508.19381