Sdg Gan Oversampling Eval

sdg-gan-oversampling-eval

Synthetic Data Generation for Fraud Detection using GANs — Charitou et al. (2021) (arXiv:2109.12546, 2021)

What this evaluates

Evaluates whether a GAN-based oversampling technique (SDG-GAN) improves binary classification performance on imbalanced tabular data compared to traditional and GAN-based baselines.

Datasets

• Credit Card Fraud Dataset — total 2492; splits: train (-1), test (-1)
• Pima Diabetes Dataset — total 768; splits: train (-1), test (-1)
• Breast Cancer Wisconsin (Diagnostic) Dataset — total 569; splits: train (-1), test (-1)
• Gambling Fraud Dataset — total 4700; splits: train (-1), test (-1)

Metrics

• algorithmic performance (primary) — range: other
  • Not explicitly defined in the provided section; refers to the overall classification performance of LR, RF, XGBoost, and MLP on the test set after training on original or SDG-GAN augmented data.

Input / output format

Input: Numeric feature vectors scaled to [0,1] with binary class labels.

Output: Binary class predictions from trained classifiers.

Scoring recipe

for dataset in [credit_card, pima, breast_cancer, gambling]:
    X_train, y_train, X_test, y_test = split(dataset)
    X_train_syn = generate_synthetic(X_train, y_train, method='SDG-GAN')
    X_train_aug = concatenate(X_train, X_train_syn)
    for clf in [LR, RF, XGBoost, MLP]:
        clf.fit(X_train_aug, y_train)
        preds = clf.predict(X_test)
        score = evaluate(preds, y_test) # Metric unspecified in text

Common pitfalls

• Min-max scaling is applied to the full dataset before train/test splitting, risking data leakage into the test set.
• The specific evaluation metric (e.g., F1, AUC, Accuracy) is not explicitly stated in the provided experimental section.
• Synthetic data is generated and concatenated with original training data, but the exact train/validation/test split ratio is not defined.

Evidence (verbatim from paper)

The following hypotheses need to be met to describe our method as successful: H1: The use of SDG-GAN to augment imbalanced datasets will improve the algorithmic performance in baseline experiments on the benchmark imbalanced datasets. H1 and H2 are tested by combining the original and synthetic datasets with the four classification algorithms, i.e. LR, RF, XGBoost and MLP, in Section VI.

Citation

@misc{charitou2021synthetic,
  title={Synthetic Data Generation for Fraud Detection using GANs},
  author={Charitou et al. (2021)},
  year={2021},
  note={arXiv:2109.12546}
}

• arXiv: 2109.12546