Tsrec Eval

tsrec-eval

TSRec: Enhancing Repeat-Aware Recommendation from a Temporal-Sequential Perspective — Quan et al. (2025) (arXiv:2506.08531, 2025)

What this evaluates

Evaluates a model's ability to perform sequential recommendation with a focus on capturing repeat-aware temporal patterns. It measures how well the model balances predicting new items versus recurring items based on user interaction history and time intervals.

Datasets

• RetailRocket — total ?; splits: train (-1), val (-1), test (-1); repo https://www.kaggle.com/retailrocket/ecommerce-dataset
• LastFM — total ?; splits: train (-1), val (-1), test (-1)
• Diginetica — total ?; splits: train (-1), val (-1), test (-1); repo http://cikm2016.cs.iupui.edu/cikm-cup

Metrics

• HR@K (primary) — range: [0, 1]
  • HR@K = (1/|U|) * Σ_{u∈U} I(rank_u ≤ K), where I is an indicator function that is 1 if the ground-truth item appears in the top-K recommendations for user u, and 0 otherwise. It measures the proportion of users for whom the correct item is ranked within the top K.
• NDCG@K — range: [0, 1]
  • NDCG@K = (1/|U|) * Σ_{u∈U} (1/log2(rank_u+1)) / (1/log2(2)), averaged over users. It measures the quality of the ranking by discounting the relevance of the ground-truth item based on its position, with higher ranks receiving less weight.

Input / output format

Input: User interaction history sequence (item IDs with timestamps) up to a maximum length L, used to predict the next item in the sequence.

Output: A ranked list of candidate items (top-K) for the next interaction, or a probability score for each item in the catalog.

Scoring recipe

def compute_metrics(predictions, ground_truth, K=20):
    hr_scores = []
    ndcg_scores = []
    for u in users:
        rank = predictions[u].index(ground_truth[u]) + 1
        hr_scores.append(1.0 if rank <= K else 0.0)
        ndcg_scores.append(1.0 / math.log2(rank + 1))
    return {
        f"HR@{K}": sum(hr_scores) / len(users),
        f"NDCG@{K}": sum(ndcg_scores) / len(users)
    }

Common pitfalls

• Negative sampling strategy differs between validation (3 negatives) and test (100 negatives) sets, which can skew ranking metrics if not handled consistently.
• Data is split chronologically by timestamp rather than randomly, requiring strict temporal ordering to prevent data leakage.
• Explicit ratings are converted to implicit feedback, meaning the evaluation focuses on ranking accuracy rather than rating prediction.

Evidence (verbatim from paper)

Following (Kang and McAuley, [2016]; Sun et al., [2019]), we use the Top-K Hit Rate (HR@K) and Top-K Normalized Discounted Cumulative Gain (NDCG@K), which are commonly used in related research (Kang and McAuley, [2016]; Sun et al., [2019]). Our reported results are based on HR@{5, 10, 20} and NDCG@{5, 10, 20}. We calculate all metrics based on item ranking and report the average scores.

Citation

@misc{quan2025tsrec,
  title={TSRec: Enhancing Repeat-Aware Recommendation from a Temporal-Sequential Perspective},
  author={Quan et al. (2025)},
  year={2025},
  note={arXiv:2506.08531}
}

• arXiv: 2506.08531