# Nllb Clip Retrieval Eval

> Probes multilingual image-text retrieval capability across low-resource languages. It evaluates how effectively the model aligns visual and textual representations when trained with limited data and frozen encoders. Use when the user wants to benchmark on XTD200, Flickr30k-200, or asks about evaluating this task. Reports R@10.

- Skill: `qhjqhj00/nllb-clip-retrieval-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/nllb-clip-retrieval-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/nllb-clip-retrieval-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/nllb-clip-retrieval-eval

---


# nllb-clip-retrieval-eval

> NLLB-CLIP -- train performant multilingual image retrieval model on a budget — Visheratin (2023) (arXiv:2309.01859, 2023)

## What this evaluates

Probes multilingual image-text retrieval capability across low-resource languages. It evaluates how effectively the model aligns visual and textual representations when trained with limited data and frozen encoders.

## Datasets

- **XTD200** — total 106246; splits: test (-1)
- **Flickr30k-200** — total ?; splits: test (-1)

## Metrics

- `R@10` **(primary)** — range: [0, 1]
  - Recall@10 measures the fraction of queries for which the ground-truth matching item appears within the top 10 retrieved results.

## Input / output format

**Input**: Image paired with a text caption in one of 201 languages.

**Output**: Retrieval ranking or similarity score between the image and text caption.

## Scoring recipe

```python
def recall_at_k(retrieved_indices, ground_truth_index, k=10):
    return 1.0 if ground_truth_index in retrieved_indices[:k] else 0.0

# Average over all queries in the dataset
total_recall = sum(recall_at_k(preds[i], gold[i]) for i in range(len(preds)))
metric_value = total_recall / len(preds)
```

## Common pitfalls

- Training the full model (unfreezing the image encoder) often fails to converge or underperforms compared to freezing the image encoder.
- Using larger text encoders can degrade performance due to insufficient data to align them with the image encoder.
- Training only the projection layers can surprisingly outperform full fine-tuning.

## Evidence (verbatim from paper)

> The smallest model variants (e.g., b-b and l-b) could not converge to get R@10 higher than 20% when we performed training of the full model.

## Citation

```bibtex
@misc{visheratin2023nllbclip,
  title={NLLB-CLIP -- train performant multilingual image retrieval model on a budget},
  author={Visheratin (2023)},
  year={2023},
  note={arXiv:2309.01859}
}
```

- arXiv: 2309.01859