# Mined Eval

> mined-eval

- Skill: `qhjqhj00/mined-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/mined-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/mined-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/mined-eval

---


# mined-eval

> MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models — Jiang et al. (2025) (arXiv:2510.19457, 2025)

## What this evaluates

Probes large multimodal models' temporal awareness and time-sensitive knowledge across six dimensions: cognition, awareness, trustworthiness, understanding, reasoning, and robustness. It evaluates how well models recall, reason about, and reject outdated or misaligned temporal facts in multimodal queries.

## Datasets

- **Mined** — total ?; splits: test (-1)

## Metrics

- `Cover Exact Match (CEM)` **(primary)** — range: [0, 1]
  - CEM is 1 if the model's output is a subset of the ground truth, else 0. Capacity is the average CEM across all subtasks.

## Input / output format

**Input**: Image and text prompt (varies across four configurations: “Question”, “Generalization Question”, “Image”, and “Generalization Image”) conveying time-sensitive knowledge queries.

**Output**: Text response from the model.

## Scoring recipe

```python
def compute_cem(model_output, ground_truth):
    return 1 if set(model_output.split()) <= set(ground_truth.split()) else 0

def compute_capacity(outputs, truths):
    return sum(compute_cem(o, t) for o, t in zip(outputs, truths)) / len(truths)
```

## Common pitfalls

- Strict exact-match (subset) requirement means minor phrasing differences yield a score of 0.
- Prompt Agreement averages scores across four prompt variations per instance, which can obscure model sensitivity to specific phrasing.
- Temporal misalignment context (especially past dates) significantly degrades performance, particularly for smaller open-source models.

## Evidence (verbatim from paper)

> In the evaluation of all subtasks, the model is considered to have correctly responded to the time-sensitive knowledge only when its output exactly matches the corresponding ground truth. Therefore, we evaluate the model’s outputs using Cover Exact Match (CEM)*(Xu et al., [2023])* score for each subtask. The model’s capacity in this dimension is defined as the average CEM score across all subtasks. CEM requires matching model’s outputs with ground truth.

## Citation

```bibtex
@misc{jiang2025mined,
  title={MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Models},
  author={Jiang et al. (2025)},
  year={2025},
  note={arXiv:2510.19457}
}
```

- arXiv: 2510.19457