Mosaic Cdsr Eval

mosaic-cdsr-eval

MOSAIC: Multi-Domain Orthogonal Session Adaptive Intent Capture for Prescient Recommendations — Bahi et al. (2026) (arXiv:2604.10147, 2026)

What this evaluates

Evaluates a model's ability to perform next-item recommendation in cross-domain sequential settings by decomposing user intent into orthogonal preference components. It probes how well the model leverages shared and domain-specific signals across multiple item categories to predict future interactions.

Datasets

• Amazon Reviews (Movie–Book) — total 2625327; splits: train (-1), val (-1), test (-1)
• Amazon Reviews (Movie–Music) — total 1138302; splits: train (-1), val (-1), test (-1)
• Douban (Movie–Book) — total 2070463; splits: train (-1), val (-1), test (-1)

Metrics

• HR@10 — range: [0, 1]
  • Hit Ratio at rank K: returns 1.0 if the ground-truth item appears in the top-K ranked list, else 0.0. Computed over the full item vocabulary.
• NDCG@10 (primary) — range: [0, 1]
  • Normalized Discounted Cumulative Gain at rank K: (1/Z) * Σ_{i=1}^K (rel_i / log2(i+1)), where rel_i=1 if the item at rank i is the ground truth, else 0. Z is the ideal DCG (1.0 for a single relevant item). Rewards higher placements of the correct item.

Input / output format

Input: Chronologically sorted user interaction history (per-domain and merged cross-domain sequences) up to timestep t-1.

Output: A ranked list of items from the entire item vocabulary, with the ground-truth next item expected at rank 1.

Scoring recipe

def compute_metrics(ranked_list, gold_item, K=10):
    rank = ranked_list.index(gold_item) + 1
    hr = 1.0 if rank <= K else 0.0
    ndcg = 0.0
    for i in range(1, K + 1):
        if i == rank:
            ndcg += 1.0 / math.log2(i + 1)
    ndcg /= 1.0  # IDCG for single relevant item
    return hr, ndcg

Common pitfalls

• Using sampled negative items for ranking instead of the full item vocabulary, which introduces sampling bias and inflates metrics.
• Ignoring the leave-one-out split protocol where the last interaction is strictly held out for testing and the second-to-last for validation.
• Failing to report mean and standard deviation over five independent random seeds with paired t-test significance checks.

Evidence (verbatim from paper)

Following standard practice in the sequential recommendation literature, we adopt Hit Ratio at rank $K$ (HR@$K$) and Normalized Discounted Cumulative Gain at rank $K$ (NDCG@$K$), with $K\in{5,10,20}$. HR@$K$ measures whether the ground-truth item appears in the top-$K$ ranked list, while NDCG@$K$ additionally rewards higher placements. All metrics are computed in the full-ranking protocol, i.e., each test item is ranked against the entire item vocabulary, to avoid sampling bias.

Citation

@misc{bahi2026mosaic,
  title={MOSAIC: Multi-Domain Orthogonal Session Adaptive Intent Capture for Prescient Recommendations},
  author={Bahi et al. (2026)},
  year={2026},
  note={arXiv:2604.10147}
}

• arXiv: 2604.10147