Cm Gnn Eval

cm-gnn-eval

Contrastive Multi-Level Graph Neural Networks for Session-based Recommendation — Wang et al. (2023) (arXiv:2311.02938, 2023)

What this evaluates

Evaluates a contrastive multi-level graph neural network for session-based recommendation by measuring its ability to predict the next item in a user session using pairwise and high-order transition patterns.

Datasets

• Tmall — total 377166; splits: train (351268), test (25898)
• Diginetica — total 780328; splits: train (719470), test (60858)
• Nowplaying — total 915128; splits: train (825304), test (89824)

Metrics

• Recall@K (primary) — range: [0, 1]
  • Standard session-based recommendation metric: fraction of sessions where the ground-truth next item appears in the top-K predicted items. Not explicitly defined in the provided text.
• NDCG@K — range: [0, 1]
  • Standard session-based recommendation metric: normalized discounted cumulative gain at rank K, measuring ranking quality. Not explicitly defined in the provided text.

Input / output format

Input: A session sequence of items [v_s,1, ..., v_s,m-1]

Output: The next item in the session v_s,m

Scoring recipe

# Not specified in provided text. Standard implementation:
def recall_at_k(predictions, gold, k):
    top_k = predictions[:k]
    return 1.0 if gold in top_k else 0.0

Common pitfalls

• Metrics are not mentioned in the provided text; standard session-rec metrics like Recall@K and NDCG@K are assumed but not confirmed.
• Preprocessing explicitly filters out sessions of length 1 and items with frequency less than 5.
• Test set is defined temporally (latest data, e.g., last week) rather than randomly split.

Evidence (verbatim from paper)

We evaluate our model on three real-world benchmark datasets, i.e., Diginetica, Tmall, and Nowplaying. ... We regard the latest data (such as the sessions of last week) as test data and the remaining data as the training set. Moreover, we augment and label both the training dataset and the test dataset by employing a sequence splitting approach for all the datasets, then we generate multiple labeled sequences with the corresponding labels, i.e., ([v_s,1], v_s,2), ([v_s,1, v_s,2], v_s,3), ..., ([v_s,1, v_s,2, ..., v_s,m-1], v_s,m).

Citation

@misc{wang2023contrastive,
  title={Contrastive Multi-Level Graph Neural Networks for Session-based Recommendation},
  author={Wang et al. (2023)},
  year={2023},
  note={arXiv:2311.02938}
}

• arXiv: 2311.02938