Topofair Fairness Eval

topofair-fairness-eval

TopoFair: Linking Topological Bias to Fairness in Link Prediction Benchmarks — Marey et al. (2026) (arXiv:2602.11802, 2026)

What this evaluates

Evaluates fairness-aware link prediction models on synthetic graphs with controlled topological biases. It probes how structural properties like assortativity, heterogeneity, and class imbalance impact fairness metrics (SP, EO) and predictive accuracy (Hit@10, AUC).

Datasets

• Opinion use case — total ?; splits: eval (-1)
• Friendship use case — total ?; splits: eval (-1)
• Collab use case — total ?; splits: eval (-1)
• Real datasets (Collab, Polblogs, Facebook) — total ?; splits: eval (-1)

Metrics

• Statistical Parity (SP) (primary) — range: [0, 1]
  • Difference in positive prediction rates between sensitive groups. Standard definition: |P(Ŷ=1|A=0) - P(Ŷ=1|A=1)|.
• Equalized Odds (EO) (primary) — range: [0, 1]
  • Difference in true positive and false positive rates between sensitive groups. Standard definition: |TPR_0 - TPR_1| + |FPR_0 - FPR_1|.
• Hit@10 — range: [0, 1]
  • Recall@10 for link prediction; fraction of test links where the true neighbor appears in the top-10 predicted links.
• AUC — range: [0, 1]
  • Area under the ROC curve measuring ranking quality of predicted link scores against binary labels.

Input / output format

Input: Graph adjacency structures and node attributes/labels used for link prediction tasks.

Output: Predicted link scores/ranks, followed by computed fairness (SP, EO) and accuracy (Hit@10, AUC) metrics per model and use case.

Scoring recipe

def compute_metrics(preds, labels, sensitive_attrs):
    hit10 = mean(top_k(preds, k=10) == labels)
    auc = roc_auc_score(labels, preds)
    sp = abs(mean(preds[sensitive_attrs==0]) - mean(preds[sensitive_attrs==1]))
    eo = abs(tpr(labels, preds, sensitive_attrs==0) - tpr(labels, preds, sensitive_attrs==1)) + \
         abs(fpr(labels, preds, sensitive_attrs==0) - fpr(labels, preds, sensitive_attrs==1))
    return hit10, auc, sp, eo

Common pitfalls

• Confounding predictive performance with fairness: weak predictive quality can artificially inflate or deflate fairness scores, so the paper explicitly verifies strong baseline accuracy first.
• Overlooking non-homophily biases: focusing solely on assortativity/homophily while ignoring heterogeneity, info unfairness, or power exp leads to incomplete fairness assessments.
• Misinterpreting bias direction: increases in heterogeneity and power exp correspond to reduced disparity (bias approaching zero), whereas higher info unfairness reflects greater inequality.

Evidence (verbatim from paper)

The results reveal clear differences across the three use cases, each exhibiting distinct patterns of variability. In the Collab setting, all methods except Flip achieve strong predictive performance, as reflected by high Hit@10 and AUC scores. Flip, while showing lower accuracy, achieves better fairness outcomes, illustrating the well-known trade-off between these objectives.

Citation

@misc{marey2026topofair,
  title={TopoFair: Linking Topological Bias to Fairness in Link Prediction Benchmarks},
  author={Marey et al. (2026)},
  year={2026},
  note={arXiv:2602.11802}
}

• arXiv: 2602.11802