Graphfusionsbr Eval

graphfusionsbr-eval

GraphFusionSBR: Denoising Multi-Channel Graphs for Session-Based Recommendation — He et al. (2026) (arXiv:2601.08497, 2026)

What this evaluates

Evaluates session-based recommendation systems by predicting the next item in a user's interaction sequence. It probes the model's ability to capture high-order item relationships and leverage external knowledge graphs for accurate, context-aware ranking.

Datasets

• Tmall — total 374178; splits: train (348766), test (25412)
• RetailRocket — total 392010; splits: train (388541), test (3469)
• KKBox — total 514513; splits: train (432689), test (81824)

Metrics

• P@10 (primary) — range: percent
  • Precision@10 measures the proportion of correctly recommended items within the top 10 predictions. Calculated as the average of binary indicators over all test sessions.
• MRR@10 — range: percent
  • Mean Reciprocal Rank@10 assesses ranking quality by taking the reciprocal of the rank of the first correct item in the top 10 list. Averaged over all test sessions.

Input / output format

Input: A session prefix sequence of item IDs [i_1, ..., i_{m-1}] and associated knowledge graph attributes (e.g., category, seller, genre).

Output: A ranked list of candidate item IDs (top K).

Scoring recipe

def compute_metrics(preds, gold, K):
    p_at_k = 0.0
    mrr_at_k = 0.0
    for p, g in zip(preds, gold):
        if g in p[:K]:
            p_at_k += 1.0
            rank = p.index(g) + 1
            mrr_at_k += 1.0 / rank
    return p_at_k / len(gold) * 100, mrr_at_k / len(gold) * 100

Common pitfalls

• Data augmentation via sequence splitting significantly inflates training set size (e.g., Tmall goes from 120k sessions to 348k sequences); failing to replicate this step will yield mismatched training scales.
• Dataset-specific filtering rules vary (e.g., RetailRocket retains sessions of length 2-4 and >4; KKBox filters tracks <1 min and enforces max order difference of 3); applying a uniform filter will corrupt the data distribution.
• Metrics are reported as percentages in tables but calculated as proportions; ensure consistent scaling when comparing results.

Evidence (verbatim from paper)

To evaluate the performance of our model, we employ two widely used metrics: Precision@$K$ (P@$K$) and Mean Reciprocal Rank@$K$ (MRR@$K$), with $K$ values set to 10 and 20. These metrics enable a robust assessment of both recommendation accuracy and ranking quality, which are essential for session-based recommendation systems. Specifically, P@$K$ measures the proportion of correctly recommended items within the top $K$ predictions.

Citation

@misc{he2026graphfusionsbr,
  title={GraphFusionSBR: Denoising Multi-Channel Graphs for Session-Based Recommendation},
  author={He et al. (2026)},
  year={2026},
  note={arXiv:2601.08497}
}

• arXiv: 2601.08497