Session Rec Eval

session-rec-eval

Prospective Preference Enhanced Mixed Attentive Model for Session-based Recommendation — Bo Peng et al. (arXiv:2206.01875, 2022)

What this evaluates

This evaluation probes a model's ability to predict the next item in a user session based on historical interactions. It measures ranking quality across multiple benchmark datasets, testing how well the model captures temporal patterns and prospective user preferences without relying on fixed recency heuristics.

Datasets

• Six session-based recommendation benchmarks (DG, GA, YC, TM, LF, NP) — total ?; splits: test (-1)

Metrics

• recall@k, MRR@k, NDCG@k (primary) — range: [0, 1]
  • Recall@k is 1 if the gold item appears in the top-k predictions, else 0. MRR@k is 1/rank if the gold item is in top-k, else 0. NDCG@k is 1/log2(rank+1) normalized by ideal DCG. Evaluated at k=10 and k=20.

Input / output format

Input: A sequence of item IDs representing a user's session history.

Output: A ranked list of candidate item IDs (top-k) for the next item prediction.

Scoring recipe

def score(predictions, gold):
    recall = 1.0 if gold in predictions[:10] else 0.0
    mrr = 0.0
    ndcg = 0.0
    for i, item in enumerate(predictions[:10]):
        if item == gold:
            mrr = 1.0 / (i + 1)
            ndcg = 1.0 / math.log2(i + 2)
            break
    return recall, mrr, ndcg

Common pitfalls

• Failing to exclude items already present in the session history from the candidate pool during evaluation.
• Using fixed recency-based heuristics (e.g., last item embedding) instead of the model's learned prospective preference estimates.
• Ignoring position embeddings, which causes significant performance drops on datasets with strong temporal patterns.

Evidence (verbatim from paper)

Table 3 presents the overall performance of different methods at recall@k, MRR@k and NDCG@k in recommending the next item.

Citation

@misc{peng2022prospective,
  title={Prospective Preference Enhanced Mixed Attentive Model for Session-based Recommendation},
  author={Bo Peng et al.},
  year={2022},
  note={arXiv:2206.01875}
}

• arXiv: 2206.01875