Ieee Cis Fraud Detection Eval

ieee-cis-fraud-detection-eval

QFNN-FFD: Quantum Federated Neural Network for Financial Fraud Detection — Innan et al. (2024) (arXiv:2404.02595, 2024)

What this evaluates

This evaluation probes a model's ability to detect financial fraud in a federated, privacy-preserving setting using quantum-enhanced neural networks. It measures classification performance on imbalanced transaction data while assessing robustness against simulated quantum hardware noise.

Datasets

• IEEE-CIS Fraud Detection — total 144233; splits: train (115386), val (28847)

Metrics

• binary classification accuracy (primary) — range: [0, 1]
  • Fraction of correctly classified transactions (fraud vs. non-fraud) out of the total validation set instances.
• MSE — range: [0, 1]
  • Mean Squared Error between predicted probabilities and actual binary labels, used as the optimization loss.

Input / output format

Input: Preprocessed numerical and categorical features from transaction and identity files, linked by TransactionID. Categorical variables are one-hot encoded, numerical variables are standardized, and the dataset is up-sampled to balance fraud/non-fraud classes.

Output: Binary classification prediction (fraud or non-fraud) per transaction instance.

Scoring recipe

def compute_metrics(predictions, labels):
    accuracy = np.mean(predictions == labels)
    mse = np.mean((predictions - labels) ** 2)
    return {'accuracy': accuracy, 'mse': mse}

Common pitfalls

• The dataset is heavily imbalanced; up-sampling during training can inflate accuracy if the validation set retains the original imbalance or is evaluated without accounting for the sampling strategy.
• Quantum noise parameters range from 0 to 1, but accuracy drops to 0 at maximum noise for some models (e.g., depolarizing), making threshold-based comparisons across noise types sensitive to the exact parameter cutoff.
• Results are averaged over 10 trials with random initialization; reporting a single run without confidence intervals may misrepresent convergence stability.

Evidence (verbatim from paper)

We focus on binary classification accuracy and MSE as key metrics. This setup is characterized by 32 initially random parameters, which are optimized through evaluations on a training set comprising 115,386 instances (80% of the total dataset of 144,233 instances) and a validation set comprising 28,847 instances, which is 20% of the total dataset.

Citation

@misc{innan2024qfnnffd,
  title={QFNN-FFD: Quantum Federated Neural Network for Financial Fraud Detection},
  author={Innan et al. (2024)},
  year={2024},
  note={arXiv:2404.02595}
}

• arXiv: 2404.02595