# Caser Sequential Rec Eval

> Evaluates a model's ability to capture sequential user behavior patterns for personalized top-N item recommendation. It probes the model's capacity to model temporal dependencies, skip behaviors, and union-level sequential patterns from historical interactions to predict future items. Use when the user wants to benchmark on MovieLens, Gowalla, Foursquare, Tmall, or asks about evaluating this task. Reports MAP.

- Skill: `qhjqhj00/caser-sequential-rec-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/caser-sequential-rec-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/caser-sequential-rec-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/caser-sequential-rec-eval

---


# caser-sequential-rec-eval

> Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding — Tang et al. (2018) (arXiv:1809.07426, 2018)

## What this evaluates

Evaluates a model's ability to capture sequential user behavior patterns for personalized top-N item recommendation. It probes the model's capacity to model temporal dependencies, skip behaviors, and union-level sequential patterns from historical interactions to predict future items.

## Datasets

- **MovieLens** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/graytowne/caser
- **Gowalla** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/graytowne/caser
- **Foursquare** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/graytowne/caser
- **Tmall** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/graytowne/caser

## Metrics

- `Precision@N` — range: [0, 1]
  - Precision@N = |R ∩ R̂_1:N| / N, where R is the test set (last 20% of user's sequence) and R̂_1:N is the top-N predicted items. Averaged over all users.
- `Recall@N` — range: [0, 1]
  - Recall@N = |R ∩ R̂_1:N| / |R|, where R is the test set and R̂_1:N is the top-N predicted items. Averaged over all users.
- `MAP` **(primary)** — range: [0, 1]
  - Mean Average Precision. AP = (Σ_{N=1}^{|R̂|} Prec@N × rel(N)) / |R̂|, where rel(N)=1 if the N-th predicted item is in R. MAP is the average AP across all users.

## Input / output format

**Input**: A user's chronological sequence of past item interactions, converted to implicit feedback (binary 1). The sequence is processed up to a maximum length L (Markov order).

**Output**: A ranked list of top-N predicted items for the user.

## Scoring recipe

```python
def compute_metrics(predictions, ground_truth, Ns=[1, 5, 10]):
    precisions, recalls, aps = [], [], []
    for pred, gold in zip(predictions, ground_truth):
        gold_set = set(gold)
        for N in Ns:
            top_n = set(pred[:N])
            precisions.append(len(top_n & gold_set) / N)
            recalls.append(len(top_n & gold_set) / len(gold))
        ap_sum = 0.0
        for k, item in enumerate(pred):
            if item in gold_set:
                ap_sum += len(set(pred[:k+1]) & gold_set) / (k + 1)
        aps.append(ap_sum / len(pred))
    return {
        'Precision@N': {N: np.mean([p for p in precisions]) for N in Ns},
        'Recall@N': {N: np.mean([r for r in recalls]) for N in Ns},
        'MAP': np.mean(aps)
    }
```

## Common pitfalls

- Data is split chronologically per user (70% train, 10% val, 20% test), not randomly shuffled.
- Cold-start users and items are filtered out (minimum 5-15 interactions depending on dataset).
- All numeric ratings are binarized to implicit feedback (1), discarding rating magnitude.
- GRU4Rec is session-based (no user ID), making direct comparison to personalized models potentially unfair for long-term preference modeling.

## Evidence (verbatim from paper)

> As in [19, 21, 29, 32], we evaluate a model by Precision@N, Recall@N, and Mean Average Precision (MAP). Given a list of top N predicted items for a user, denoted R̂_1:N, and the last 20% of actions in her/his sequence (i.e., denoted R (i.e., the test set), Precision@N and Recall@N are computed by Prec@N = |R ∩ R̂_1:N| / N, Recall@N = |R ∩ R̂_1:N| / |R|. We report the average of these values of all users. N∈{1,5,10} The Average Precision (AP) is defined by AP = (sum_{N=1}^{|R̂|} Prec@N × rel(N)) / |R̂|, where rel(N) = 1 if the N-th item in R̂ is in R. The Mean Average Precision (MAP) is the average of AP for all users.

## Citation

```bibtex
@misc{tang2018caser,
  title={Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding},
  author={Tang et al. (2018)},
  year={2018},
  note={arXiv:1809.07426}
}
```

- arXiv: 1809.07426