# Vdr Compression Eval

> Evaluates multi-vector visual document retrieval (VDR) models under varying compression ratios. It measures how well pruning and merging strategies maintain retrieval accuracy while reducing storage and computational overhead. Use when the user wants to benchmark on ViDoRe-V1, ViDoRe-V2, JinaVDR-Bench, REAL-MM-RAG, ViDoSeek, MMLongBench-Doc, or asks about evaluating this task. Reports nDCG@5.

- Skill: `qhjqhj00/vdr-compression-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/vdr-compression-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/vdr-compression-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/vdr-compression-eval

---


# vdr-compression-eval

> Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework — Yan et al. (2026) (arXiv:2602.19549, 2026)

## What this evaluates

Evaluates multi-vector visual document retrieval (VDR) models under varying compression ratios. It measures how well pruning and merging strategies maintain retrieval accuracy while reducing storage and computational overhead.

## Datasets

- **ViDoRe-V1** — total ?; splits: test (-1); repo https://github.com/illuin-tech/vidore-benchmark
- **ViDoRe-V2** — total ?; splits: test (-1); repo https://github.com/illuin-tech/vidore-benchmark
- **JinaVDR-Bench** — total ?; splits: test (-1)
- **REAL-MM-RAG** — total ?; splits: test (-1)
- **ViDoSeek** — total ?; splits: test (-1)
- **MMLongBench-Doc** — total ?; splits: test (-1)

## Metrics

- `nDCG@5` **(primary)** — range: [0, 1]
  - Normalized Discounted Cumulative Gain at rank 5. It measures the quality of the ranked list of retrieved documents by discounting relevance by position, normalized by the ideal DCG.

## Input / output format

**Input**: Visual document images (or pages) and a natural language query.

**Output**: A ranked list of retrieved document IDs or embeddings.

## Scoring recipe

```python
def compute_ndcg_at_k(relevant_docs, retrieved_docs, k=5):
    dcg = 0.0
    for i, doc_id in enumerate(retrieved_docs[:k]):
        if doc_id in relevant_docs:
            dcg += 1.0 / math.log2(i + 2)
    num_relevant = min(len(relevant_docs), k)
    idcg = sum(1.0 / math.log2(i + 2) for i in range(num_relevant))
    return dcg / idcg if idcg > 0 else 0.0
```

## Common pitfalls

- Compression ratios vary per document for adaptive methods; average compression is reported rather than a fixed per-document ratio.
- Evaluation focuses on offline indexing latency and storage reduction, not retrieval latency.
- Performance is evaluated across multiple base models (ColQwen2.5, ColNomic, Jina-v4), so results are not model-agnostic.

## Evidence (verbatim from paper)

> Following standard VDR practices, we adopt nDCG@5 as the primary evaluation metric. Our experimental validation is performed on a comprehensive suite of six representative VDR benchmarks, totaling 29 distinct datasets (more details in Appendix [E]). These include ViDoRe-V1 (Faysse et al., [2024]), ViDoRe-V2 (macé2025vidorebenchmarkv2raising), JinaVDR-Bench (Günther et al., [2025]), REAL-MM-RAG (Wasserman et al., [2025]), ViDoSeek (Wang et al., [2025b]), and MMLongBench-Doc (Ma et al., [2024b]).

## Citation

```bibtex
@misc{yan2026sculpting,
  title={Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework},
  author={Yan et al. (2026)},
  year={2026},
  note={arXiv:2602.19549}
}
```

- arXiv: 2602.19549