Clicktok Eval

clicktok-eval

Clicktok: Click Fraud Detection using Traffic Analysis — Nagaraja et al. (2019) (arXiv:1903.00733, 2019)

What this evaluates

Evaluates the ability to detect fraudulent ad clicks (clickspam) by analyzing temporal reuse patterns in organic clickstreams. It tests both passive traffic analysis and active bait-click injection strategies to distinguish legitimate user behavior from automated or malware-driven fraud.

Datasets

• University Network Click Traffic Dataset — total 217334190; splits: evaluation (217334190)

Metrics

• FPR (primary) — range: percent
  • False Positive Rate: the fraction of legitimate clicks incorrectly reported as fraudulent. Calculated as FP / (FP + TN).
• TPR — range: percent
  • True Positive Rate: the fraction of fraudulent clicks correctly detected. Calculated as TP / (TP + FN).

Input / output format

Input: Time-series click traffic aggregated into a matrix where each row represents a source IP address per day, and each column represents a 5-minute interval (288 columns/day). Each cell contains the total click count. Additional features per click include ad URL, ad server IP, referrer URL, source IP, User-Agent string, and timestamp.

Output: Binary classification per click or per time-window: predicted as legitimate (0) or clickspam/fraudulent (1). For active bait-click defense, outputs the fraction of fraud clicks per time-window based on pattern activation weights.

Scoring recipe

def compute_tpr_fpr(predictions, gold_labels):
    tp = sum(1 for p, g in zip(predictions, gold_labels) if p == 1 and g == 1)
    fp = sum(1 for p, g in zip(predictions, gold_labels) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(predictions, gold_labels) if p == 0 and g == 1)
    tn = sum(1 for p, g in zip(predictions, gold_labels) if p == 0 and g == 0)
    tpr = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    fpr = fp / (fp + tn) if (fp + tn) > 0 else 0.0
    return {'TPR': tpr, 'FPR': fpr}

Common pitfalls

• Confusing the passive (mimicry) and active (bait-click) evaluation setups, which require different matrix initialization and pattern isolation methods.
• Misinterpreting the temporal granularity: the evaluation bins clicks into 5-minute intervals per source IP per day, not per individual click event.
• Overlooking that the dataset is synthetically generated by exposing legitimate university traffic to known click malware, meaning real-world distribution shift may affect generalization.

Evidence (verbatim from paper)

The FP and TP rates are the fraction of legitimate clicks reported as fraudulent clicks, and the fraction of fraudulent clicks detected, respectively.

Citation

@misc{nagaraja2019clicktok,
  title={Clicktok: Click Fraud Detection using Traffic Analysis},
  author={Nagaraja et al. (2019)},
  year={2019},
  note={arXiv:1903.00733}
}

• arXiv: 1903.00733