# Sasrec Sequential Rec Eval

> Evaluates a model's ability to predict the next item in a user's interaction sequence based on historical behavior. It probes the model's capacity to capture long-range dependencies and adapt to varying data sparsity across different domains. Use when the user wants to benchmark on Amazon (Beauty), Amazon (Games), Steam, MovieLens-1M, or asks about evaluating this task. Reports Recall@K.

- Skill: `qhjqhj00/sasrec-sequential-rec-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/sasrec-sequential-rec-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/sasrec-sequential-rec-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/sasrec-sequential-rec-eval

---


# sasrec-sequential-rec-eval

> Self-Attentive Sequential Recommendation — Wang-Cheng Kang et al. (2018) (arXiv:1808.09781, 2018)

## What this evaluates

Evaluates a model's ability to predict the next item in a user's interaction sequence based on historical behavior. It probes the model's capacity to capture long-range dependencies and adapt to varying data sparsity across different domains.

## Datasets

- **Amazon (Beauty)** — total ?; splits: train (-1), val (-1), test (-1)
- **Amazon (Games)** — total ?; splits: train (-1), val (-1), test (-1)
- **Steam** — total 7793069; splits: train (-1), val (-1), test (-1)
- **MovieLens-1M** — total 1000000; splits: train (-1), val (-1), test (-1)

## Metrics

- `Recall@K` **(primary)** — range: [0, 1]
  - Standard sequential recommendation metric; fraction of users for whom the ground-truth next item appears in the top-K predicted items.

## Input / output format

**Input**: A sequence of user-item interactions (reviews/ratings) ordered by timestamp, including all training actions and the validation action.

**Output**: A single predicted item (the next interaction in the sequence).

## Scoring recipe

```python
def evaluate(predictions, ground_truth, K):
    hits = 0
    for pred_list, true_item in zip(predictions, ground_truth):
        if true_item in pred_list[:K]:
            hits += 1
    return hits / len(ground_truth)
```

## Common pitfalls

- Users and items with fewer than 5 related actions are discarded, significantly altering dataset statistics.
- During testing, the input sequence explicitly includes the validation action alongside training actions, not just the training history.
- Splits are chronological per-user (leave-one-out), not random, so temporal leakage must be avoided.

## Evidence (verbatim from paper)

> We followed the same preprocessing procedure from [1], [19], [21]. For all datasets, we treat the presence of a review or rating as implicit feedback (i.e., the user interacted with the item) and use timestamps to determine the sequence order of actions. We discard users and items with fewer than 5 related actions. For partitioning, we split the historical sequence $S^u$ for each user $u$ into three parts: (1) the most recent action $S_{|S^u|}^u$ for testing, (2) the second most recent action $S_{|S^u| - 1}^u$ for validation, (3) all remaining actions for training. Note that during testing, the input sequences contain training actions and the validation action.

## Citation

```bibtex
@misc{kang2018sasrec,
  title={Self-Attentive Sequential Recommendation},
  author={Wang-Cheng Kang et al. (2018)},
  year={2018},
  note={arXiv:1808.09781}
}
```

- arXiv: 1808.09781