Creditprint Eval

creditprint-eval

CreditPrint: Credit Investigation via Geographic Footprints by Deep Learning — Han et al. (2019) (arXiv:1910.08734, 2019)

What this evaluates

Evaluates a model's ability to predict user creditworthiness based on geographic mobility footprints. It probes whether spatiotemporal visitation patterns and region-level credit signals can reliably distinguish users who pay their mobile phone bills from those who do not.

Datasets

• Hangzhou user mobility dataset — total 5000; splits: train (-1), val (-1), test (-1)

Metrics

• AUC (primary) — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve, measuring the probability that a randomly chosen positive instance ranks higher than a randomly chosen negative instance across all classification thresholds.

Input / output format

Input: Sequences of 1km×1km geographic regions visited by a user over a one-month period.

Output: Binary credit label indicating whether the user pays their mobile phone bill (1) or not (0).

Scoring recipe

def compute_auc(y_true, y_pred):
    # y_true: ground truth binary labels (1 = pays bill, 0 = does not)
    # y_pred: predicted probabilities of paying the bill
    # Compute Area Under the Receiver Operating Characteristic Curve
    from sklearn.metrics import roc_auc_score
    return roc_auc_score(y_true, y_pred)

Common pitfalls

• The dataset is proprietary and collected from a single mobile operator in Hangzhou, China, limiting generalizability to other regions or operators.
• The credit label is binary (mobile bill payment status) rather than a comprehensive financial credit score, which restricts direct comparison with standard credit scoring benchmarks.
• Manual feature baselines rely on a fixed set of 6 handcrafted mobility features, which may not represent the full state-of-the-art in feature engineering.

Evidence (verbatim from paper)

We adopt the widely-used binary classification metric, AUC (Area Under Curve) to verify the effectiveness of user credit investigation.

Citation

@misc{han2019creditprint,
  title={CreditPrint: Credit Investigation via Geographic Footprints by Deep Learning},
  author={Han et al. (2019)},
  year={2019},
  note={arXiv:1910.08734}
}

• arXiv: 1910.08734