# Recsys Ranking

> Recommender & Ranking Systems

- Skill: `sanjeevrg89/recsys-ranking` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add sanjeevrg89/recsys-ranking`
- Raw SKILL.md: https://api.skillmd.com/api/skills/sanjeevrg89/recsys-ranking/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: sanjeevrg89 (https://skillmd.com/u/sanjeevrg89)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/sanjeevrg89/recsys-ranking

---


# Recommender & Ranking Systems

Apply the judgment of an engineer who has owned a recommendation or ranking surface in production at
scale for years — where a 0.5% relevance win is millions of dollars, the offline metric lies, and the
feedback loop will quietly eat your feed if you optimize the wrong objective.

## How to use this skill

1. **Read `recsys-ranking-guide.md`** in this directory — the full reference (funnel, models, features,
   evaluation, production concerns, anti-patterns). Apply it to the task.
2. For a concrete two-tower + ranking funnel sketch and an offline+online eval plan for a ranking
   change, read **`examples.md`**.
3. Match the surrounding system's conventions (feature store, serving stack, framework). Apply the
   correctness rules — train/serve consistency, position-bias-aware training, honest online eval —
   regardless. The ecosystem moves fast (it is 2026); flag fast-moving claims and verify current docs.

## The essentials (full detail in `recsys-ranking-guide.md`)

- **Stage the funnel.** Millions of items → ~hundreds → ~tens, under a tight latency budget. Each stage
  trades recall for precision: cheap+high-recall retrieval, mid pre-ranking, expensive precise ranking,
  then re-ranking and policy. Don't try to score the whole corpus with your heavy model.
- **Retrieval = embeddings + ANN, plus heuristic sources.** Two-tower/dual-encoder with ANN
  (`[[rag-vector-databases]]`) is the workhorse; complement with collaborative filtering (MF/ALS),
  graph, and rule-based sources. Blend multiple retrievers — no single one covers the space.
- **Negative sampling is the whole game in retrieval.** In-batch negatives + sampled-softmax with a
  log-Q correction; add **hard negatives** for precision. Cheap negatives → models that can't tell
  good from mediocre.
- **Ranking: rich features beat clever models, up to a point.** GBDTs are a strong baseline; deep
  rankers (Wide&Deep → DeepFM/DCN-v2 → DLRM → transformer rankers) win when you have huge sparse
  features and feature crosses worth learning. Pick for *your* data and latency, not the leaderboard.
- **Most surfaces are multi-objective.** Optimize engagement *and* satisfaction *and* diversity, not
  CTR alone. Use multi-task architectures (MMoE/PLE) and combine task heads with a tuned value model;
  CTR-only optimization produces degenerate clickbait feeds.
- **Train/serve skew is the #1 silent killer.** The same feature transforms, the same feature store,
  point-in-time-correct training labels (`[[data-engineering-feature-stores]]`). Log features at
  serving time and train on *those*.
- **Offline metrics (nDCG, MAP, MRR, recall@k, AUC) only rank candidates; online A/B decides.** Expect
  an offline-online gap. Watch CTR *and* long-term/north-star metrics; use counterfactual/off-policy
  evaluation before you ship to a live test.
- **Position bias will silently train your model to predict layout, not relevance.** Model position
  (e.g. position-as-feature, IPS) or you learn a self-fulfilling prophecy.
- **You need exploration.** Pure exploitation collapses into filter bubbles and starves new items
  (cold start) and the model of counterfactual data. Bandits / ε-exploration / dithering.
- **Calibration matters when scores feed downstream** (ads auctions, blending). A well-ranked but
  miscalibrated pCTR breaks bidding and multi-objective blends.

## Related skills

- `[[ml-system-design]]` — the broader ML-system design frame this fits inside.
- `[[rag-vector-databases]]` — ANN indexes (HNSW, IVF-PQ, ScaNN) for embedding retrieval.
- `[[data-engineering-feature-stores]]` — feature store, point-in-time joins, train/serve consistency.
- `[[ml-evaluation-evals]]` — ranking/relevance metrics, A/B testing, off-policy evaluation.
- `[[ml-frameworks]]` — PyTorch/JAX for training rankers and embedding models.
- `[[serving-frameworks]]` — low-latency model serving for the ranking stage.
- `[[aiml-on-kubernetes]]` — running training and serving on K8s/GKE at scale.