Transboost Eval

transboost-eval

TransBoost: A Boosting-Tree Kernel Transfer Learning Algorithm for Improving Financial Inclusion — Sun et al. (2021) (arXiv:2112.02365, 2021)

What this evaluates

Evaluates a boosting-tree kernel transfer learning algorithm for financial risk prediction and fraud detection under domain distribution shifts and data sparsity. It measures how well the model adapts from a source domain to a target domain with limited labeled samples, while maintaining computational efficiency and interpretability.

Datasets

• Tencent Mobile Payment Dataset — total ?; splits: train (1537573), test (39735)
• LendingClub Dataset — total ?; splits: train (6080), test (3000)
• Wine Quality Dataset — total ?; splits: train (4418), test (960)

Metrics

• AUC (primary) — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve. Computed by ranking predicted fraud/default probabilities against binary ground-truth labels and calculating the area under the ROC curve.
• Runtime — range: seconds
  • Wall-clock training time measured in seconds. Averaged over 10 independent runs with different training sample fractions.

Input / output format

Input: Tabular feature vectors (e.g., 2,714 features for Tencent, 110 for LendingClub, 11 for Wine) with binary labels indicating fraud/default status or quality threshold.

Output: Predicted probability of the positive class (fraud/default), used to compute AUC and threshold-dependent metrics like loan approval ratio.

Scoring recipe

def compute_auc(y_true, y_pred_proba):
    sorted_indices = np.argsort(y_pred_proba)[::-1]
    y_true_sorted = np.array(y_true)[sorted_indices]
    tpr = np.cumsum(y_true_sorted) / np.sum(y_true_sorted)
    fpr = np.cumsum(1 - y_true_sorted) / np.sum(1 - y_true_sorted)
    auc = np.trapz(tpr, fpr)
    return auc

Common pitfalls

• The evaluation uses a domain adaptation setup where the target domain has labeled training data, but some baselines are originally designed for unsupervised DA. The authors fine-tune them with true target labels for fairness, which may inflate baseline performance.
• Data sparsity is simulated by randomly setting features to NULL, which may not reflect real-world missing data mechanisms or feature correlations.
• The Tencent dataset is private and sampled specifically for testing; results may not generalize to the full production distribution.

Evidence (verbatim from paper)

We use a fix-sized testing dataset to evaluate the model performance with AUC. The AUCs of TransBoost and other baseline methods on LendingClub, Wine quality, and Tencent mobile payment datasets are shown in Tables 2, 3, and Figure 2, respectively. Note that fewer results are shown in Figure 2 because some of the traditional benchmark models ([1, 3, 5, 7]) cannot handle the large volume and high sparsity of the Tencent mobile payment dataset.

Citation

@misc{sun2021transboost,
  title={TransBoost: A Boosting-Tree Kernel Transfer Learning Algorithm for Improving Financial Inclusion},
  author={Sun et al. (2021)},
  year={2021},
  note={arXiv:2112.02365}
}

• arXiv: 2112.02365