# Inst It Bench Eval

> Evaluates a model's ability to perform fine-grained, instance-level understanding on images and videos. It probes spatial-temporal grounding, multi-level annotation comprehension (captions, temporal changes), and multiple-choice question answering over explicitly prompted visual regions. Use when the user wants to benchmark on Inst-IT Bench, or asks about evaluating this task. Reports average score.

- Skill: `qhjqhj00/inst-it-bench-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/inst-it-bench-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/inst-it-bench-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/inst-it-bench-eval

---


# inst-it-bench-eval

> INST-IT: Boosting Instance Understanding via Explicit Visual Prompt Instruction Tuning — Peng et al. (2024) (arXiv:2412.03565, 2024)

## What this evaluates

Evaluates a model's ability to perform fine-grained, instance-level understanding on images and videos. It probes spatial-temporal grounding, multi-level annotation comprehension (captions, temporal changes), and multiple-choice question answering over explicitly prompted visual regions.

## Datasets

- **Inst-IT Bench** — total ?; splits: test (-1)

## Metrics

- `average score` **(primary)** — range: percent
  - Percentage of correctly answered multiple-choice questions on the Inst-IT Bench dataset. Calculated as (number of correct predictions / total number of instances) * 100.

## Input / output format

**Input**: Images or video frames with explicit visual prompts (e.g., bounding boxes or highlighted regions) overlaid, accompanied by an instruction prompt (e.g., multiple-choice question or open-ended query).

**Output**: Text response containing the selected option letter/answer for multiple-choice questions, or a generated caption/QA response for open-ended tasks.

## Scoring recipe

```python
def compute_average_score(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p.strip().upper() == g.strip().upper())
    return (correct / len(gold)) * 100
```

## Common pitfalls

- The benchmark explicitly uses visual prompts (bounding boxes/regions) in the input, which differs from standard image/video benchmarks that only provide raw pixels.
- Inst-IT Bench contains both image (Inst-IT-I) and video (Inst-IT-V) splits evaluated in multiple-choice format; results should not be conflated with open-ended or zero-shot benchmarks like ViP-Bench or RefCOCOg.
- The model is trained via a continuous instruction-tuning paradigm with frozen vision encoder layers; evaluating without this specific training recipe will yield baseline LLaVA-NeXT performance, not the proposed method.

## Evidence (verbatim from paper)

> We conduct extensive evaluations on Inst-IT Bench. The results in [Tab. 2] show that with instruction tuning using Inst-IT Dataset, our models achieve a significant improvement of nearly 20% on average score, validating the effectiveness of Inst-IT.

## Citation

```bibtex
@misc{peng2024instit,
  title={INST-IT: Boosting Instance Understanding via Explicit Visual Prompt Instruction Tuning},
  author={Peng et al. (2024)},
  year={2024},
  note={arXiv:2412.03565}
}
```

- arXiv: 2412.03565