Lfree Da Eval

lfree-da-eval

LFreeDA: Label-Free Drift Adaptation for Windows Malware Detection — Li et al. (2025) (arXiv:2511.14963, 2025)

What this evaluates

Evaluates a malware detection model's ability to adapt to natural concept drift over time using a rolling monthly update setup on real-world Windows malware binaries.

Datasets

• MB-24+ — total ?; splits: source_train (-1), source_test (-1), target_train (-1), target_test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Standard classification accuracy: fraction of correctly predicted labels out of total samples.
• macro-F1 — range: [0, 1]
  • Macro-averaged F1 score: harmonic mean of precision and recall computed per class (malware, benign) and then averaged.

Input / output format

Input: Windows PE binaries represented as Control-Flow Graphs (CFGs) or image-based representations (DIReps).

Output: Binary classification label (malware vs. benign).

Scoring recipe

def score(predictions, gold):
    acc = sum(p == g for p, g in zip(predictions, gold)) / len(gold)
    classes = [0, 1]
    f1s = []
    for c in classes:
        tp = sum(1 for p, g in zip(predictions, gold) if p == c and g == c)
        fp = sum(1 for p, g in zip(predictions, gold) if p == c and g != c)
        fn = sum(1 for p, g in zip(predictions, gold) if p != c and g == c)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        f1s.append(f1)
    macro_f1 = sum(f1s) / len(f1s)
    return acc, macro_f1

Common pitfalls

• Temporal leakage must be strictly avoided; source and target data are partitioned by month, with June skipped to ensure clean separation.
• Benign samples lack temporal labels and are assumed distributionally stable, but CFG extraction fails for some files due to disassembly errors.
• Pseudo-labels are generated without ground truth during adaptation, requiring confidence thresholding and outlier detection to filter noise.

Evidence (verbatim from paper)

Performance is measured by accuracy and macro-F1, and all results are averaged over five independent runs.

Citation

@misc{li2025lfreeeda,
  title={LFreeDA: Label-Free Drift Adaptation for Windows Malware Detection},
  author={Li et al. (2025)},
  year={2025},
  note={arXiv:2511.14963}
}

• arXiv: 2511.14963