Temporal Graph Anomaly Eval

temporal-graph-anomaly-eval

Temporal Graphs Anomaly Emergence Detection: Benchmarking For Social Media Interactions — Lazebnik et al. (2023) (arXiv:2307.05268, 2023)

What this evaluates

Evaluates the ability of various data-driven models to detect emerging anomalies in temporal graphs derived from social media interactions. It probes how well different architectures generalize across different social platforms and remain robust to parameter variations and temporal/spatial shifts.

Datasets

• Twitter — total ?; splits: (unstated)
• Facebook — total ?; splits: (unstated)

Metrics

• weighted F1 score (primary) — range: [0, 1]
  • Harmonic mean of precision and recall, weighted by the support (number of true instances) for each class. Reported as mean ± standard deviation over 100 instances per dataset.

Input / output format

Input: Temporal graph data representing social media interactions.

Output: Anomaly scores or binary labels indicating the presence of emerging anomalies.

Scoring recipe

def weighted_f1(y_true, y_pred, labels):
    precisions, recalls, supports = precision_recall_fscore_support(y_true, y_pred, labels=labels, average=None)
    weights = supports / supports.sum()
    f1s = 2 * (precisions * recalls) / (precisions + recalls + 1e-8)
    return np.average(f1s, weights=weights)

Common pitfalls

• Performance is highly dataset-dependent; Facebook consistently yields lower scores than Twitter across all models.
• No single model consistently outperforms others across both datasets or sensitivity tests, indicating poor generalizability.
• Sensitivity to prediction lag and temporal concept drift varies significantly by model architecture, complicating hyperparameter tuning.

Evidence (verbatim from paper)

Fig. 2 summarizes the main results obtained where Figs. 2(a) and 2(b) show the weighted $F_{1}$ score of each model for the Twitter and Facebook datasets, respectively. The results are shown as the mean $\pm$ standard deviation of $n=100$ instances for each dataset.

Citation

@misc{lazebnik2023benchmarking,
  title={Temporal Graphs Anomaly Emergence Detection: Benchmarking For Social Media Interactions},
  author={Lazebnik et al. (2023)},
  year={2023},
  note={arXiv:2307.05268}
}

• arXiv: 2307.05268