Multiview Cir Eval

multiview-cir-eval

FashionMV: Product-Level Composed Image Retrieval with Multi-View Fashion Data — Yuan et al. (2026) (arXiv:2604.10297, 2026)

What this evaluates

This benchmark evaluates a model's ability to perform product-level composed image retrieval (CIR) in fashion e-commerce, specifically handling multi-view product images and short modification queries. It probes the model's capacity to align visual perception with textual reasoning across multiple views while filtering out irrelevant gallery items.

Datasets

• DeepFashion — total ?; splits: val (-1)
• Fashion200K — total ?; splits: val (-1)
• FashionGen-val — total ?; splits: val (-1)

Metrics

• Recall@5 (primary) — range: [0, 1]
  • The proportion of ground-truth target products that appear in the top-5 retrieved results from the gallery.
• Recall@10 — range: [0, 1]
  • The proportion of ground-truth target products that appear in the top-10 retrieved results from the gallery.

Input / output format

Input: A query consisting of one or more reference product images (multi-view), a short modification text describing the desired change, and a document gallery of candidate product images.

Output: A ranked list of candidate products from the gallery, or binary relevance labels for each candidate to compute top-k recall.

Scoring recipe

def compute_recall_at_k(predictions, gold_indices, k=5):
    top_k = predictions[:k]
    hits = sum(1 for idx in top_k if idx in gold_indices)
    return hits / len(gold_indices)

Common pitfalls

• Using single-image queries instead of multi-view inputs, which the paper notes is fundamentally inadequate for this task.
• Relying on post-hoc aggregation (MeanPool/MaxSim) instead of native joint encoding for models that support it, as averaging can dilute view-specific details.
• Failing to restrict queries to short modification text only, which the authors emphasize reflects realistic user scenarios.

Evidence (verbatim from paper)

We evaluate using short modification text only, as it reflects the realistic user query scenario (§3.4), and report Recall@5 and Recall@10.

Citation

@misc{yuan2026fashionmv,
  title={FashionMV: Product-Level Composed Image Retrieval with Multi-View Fashion Data},
  author={Yuan et al. (2026)},
  year={2026},
  note={arXiv:2604.10297}
}

• arXiv: 2604.10297