Fraud Detection Accuracy Eval

fraud-detection-accuracy-eval

Enhancing Customer Contact Efficiency with Graph Neural Networks in Credit Card Fraud Detection Workflow — Huo et al. (2025) (arXiv:2504.02275, 2025)

What this evaluates

Evaluates a graph neural network classifier's ability to determine whether credit card transactions require customer contact, aiming to optimize fraud detection workflows and reduce false positives.

Datasets

• Custom credit card transaction dataset — total ?; splits: train (-1), test (-1)

Metrics

• accuracy (primary) — range: [0, 1] | percent
  • Standard classification accuracy: the ratio of correctly classified transactions to the total number of transactions. The paper reports a final value of 0.9988% (likely a typographical error for 99.88%, as it states the metric 'highly approach[es] 100%').

Input / output format

Input: Heterogeneous graph representation of transaction data, with node and edge features derived from users, devices, locations, and transactional relationships.

Output: Binary classification label indicating whether a transaction requires customer contact (positive) or not (negative).

Scoring recipe

def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

Common pitfalls

• The reported accuracy of 0.9988% appears to be a typographical error for 99.88%, as the text explicitly states it 'highly approach[es] 100%'.
• The evaluation lacks details on cross-validation, external test sets, or specific thresholds for converting model probabilities into binary contact decisions.
• Class imbalance is addressed via focal loss during training, but no corresponding evaluation metrics (e.g., precision, recall, F1) are reported to assess false positive reduction.

Evidence (verbatim from paper)

The dataset was split into training and testing sets with an 80:20 ratio. The final accuracy can achieve 0.9988%, which is highly approach 100%.

Citation

@misc{huo2025enhancing,
  title={Enhancing Customer Contact Efficiency with Graph Neural Networks in Credit Card Fraud Detection Workflow},
  author={Huo et al. (2025)},
  year={2025},
  note={arXiv:2504.02275}
}

• arXiv: 2504.02275