Clickbait Mitigation Eval

clickbait-mitigation-eval

Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue — Wenjie Wang et al. (2020) (arXiv:2009.09945, 2020)

What this evaluates

This evaluation probes a recommender system's ability to mitigate clickbait by measuring performance exclusively on user interactions that result in positive post-click feedback (likes), rather than raw click-through rates.

Datasets

• Unspecified in provided section — total ?; splits: test (-1)

Metrics

• post-click satisfaction (likes) (primary) — range: [0, 1]
  • Recommendation performance is computed exclusively on the subset of test interactions where the user provided positive post-click feedback (likes). Interactions lacking post-click feedback are excluded from evaluation.

Input / output format

Input: User features $u$ and item features $i=(e,t)$, where $e$ denotes exposure features (e.g., title, cover) and $t$ denotes content features.

Output: A predicted preference score $Y_{u,i} = s_{\bar{\theta}}(u,i)$ used to rank items.

Scoring recipe

# 1. Filter test interactions to keep only those with positive post-click feedback (likes)
liked_interactions = [(u, i) for (u, i, y) in test_data if y == 'like']
# 2. Exclude interactions lacking post-click feedback
# 3. Compute standard recommendation performance metric (e.g., AUC/NDCG) on the filtered subset
metric_value = compute_recommender_metric(model, liked_interactions)

Common pitfalls

• Conventional recommender evaluations treat all clicks as positive signals; this protocol explicitly excludes clicks without post-click feedback to isolate clickbait effects.
• Evaluating on raw click-through rate (CTR) will overestimate performance on clickbait items; this protocol requires post-click satisfaction signals.

Evidence (verbatim from paper)

Distinct from the conventional recommender evaluation that treats all clicks in the testing period as positive samples*(Wei et al., 2019); He and McAuley, 2016), we evaluate recommendation performance only over clicks that end with positive post-click feedback (i.e., likes)(Wen et al., 2020)*. We do not use the clicks that lack post-click feedback due to the unawareness of user satisfaction.

Citation

@misc{wang2020clickbait,
  title={Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue},
  author={Wenjie Wang et al. (2020)},
  year={2020},
  note={arXiv:2009.09945}
}

• arXiv: 2009.09945