Wpgrec Eval

wpgrec-eval

WPGRec: Wavelet Packet Guided Graph Enhanced Sequential Recommendation — Liu et al. (2026) (arXiv:2604.21305, 2026)

What this evaluates

Evaluates a model's ability to perform sequential recommendation by predicting the next item a user will interact with based on their chronological interaction history. It probes the model's capacity to capture temporal dynamics and collaborative filtering signals while ranking items against a full candidate set.

Datasets

• MovieLens-1M* — total ?; splits: train (-1), val (-1), test (-1)
• Amazon-Beauty — total ?; splits: train (-1), val (-1), test (-1)
• Amazon-Sports — total ?; splits: train (-1), val (-1), test (-1)
• LastFM (HetRec 2011) — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• HR@10 (primary) — range: [0, 1]
  • Hit Ratio at K equals 1 if the ground-truth item appears in the top-K predicted items, otherwise 0.
• HR@20 — range: [0, 1]
  • Hit Ratio at K equals 1 if the ground-truth item appears in the top-K predicted items, otherwise 0.
• NDCG@10 — range: [0, 1]
  • Normalized Discounted Cumulative Gain at K computes the discounted gain at the predicted rank of the ground-truth item, normalized by the ideal DCG (which is 1 for a single relevant item), capped at K.
• NDCG@20 — range: [0, 1]
  • Normalized Discounted Cumulative Gain at K computes the discounted gain at the predicted rank of the ground-truth item, normalized by the ideal DCG (which is 1 for a single relevant item), capped at K.

Input / output format

Input: A user's chronological sequence of previously interacted items (training history), optionally padded or truncated to a fixed maximum length.

Output: A score or ranking for every item in the catalog, used to produce a top-K ranked list excluding training items.

Scoring recipe

def compute_metrics(predictions, ground_truth, k_values=[10, 20]):
    ranked_items = np.argsort(predictions)[::-1]
    results = {}
    for k in k_values:
        top_k = ranked_items[:k]
        hit = 1 if ground_truth in top_k else 0
        results[f'HR@{k}'] = hit
        if hit:
            rank_pos = np.where(ranked_items == ground_truth)[0][0] + 1
            results[f'NDCG@{k}'] = 1.0 / np.log2(rank_pos + 1)
        else:
            results[f'NDCG@{k}'] = 0.0
    return results

Common pitfalls

• Using sampled negatives for training or evaluation instead of full-softmax/full-ranking over the entire item set.
• Failing to exclude items present in the user's training history from the candidate set during evaluation, which artificially inflates metrics.
• Not splitting data chronologically, which leaks future interactions into the training set.

Evidence (verbatim from paper)

We use a full-ranking Top-$K$ protocol, where the ground-truth test item is ranked against the entire item set (excluding items seen in the user’s training history). We report HR@10, HR@20, NDCG@10, and NDCG@20. No sampled negatives are used in either training or evaluation: training uses full-softmax cross-entropy over the entire item set, and evaluation uses full-ranking over all candidate items.

Citation

@misc{liu2026wpgrec,
  title={WPGRec: Wavelet Packet Guided Graph Enhanced Sequential Recommendation},
  author={Liu et al. (2026)},
  year={2026},
  note={arXiv:2604.21305}
}

• arXiv: 2604.21305