# Silicone Prep Anomaly Eval

> Evaluates multimodal vision-language models' ability to detect context-dependent visual anomalies in robotic scientific laboratory workflows using first-person imagery and stage-specific textual prompts. Use when the user wants to benchmark on Silicone Preparation Workflow, or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/silicone-prep-anomaly-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/silicone-prep-anomaly-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/silicone-prep-anomaly-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/silicone-prep-anomaly-eval

---


# silicone-prep-anomaly-eval

> A VLM-based Method for Visual Anomaly Detection in Robotic Scientific Laboratories — Lin et al. (2025) (arXiv:2506.05405, 2025)

## What this evaluates

Evaluates multimodal vision-language models' ability to detect context-dependent visual anomalies in robotic scientific laboratory workflows using first-person imagery and stage-specific textual prompts.

## Datasets

- **Silicone Preparation Workflow** — total 1001; splits: test (1001)

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Binary classification accuracy; proportion of correctly classified normal vs. abnormal images. Exact formula not detailed in this section.

## Input / output format

**Input**: First-person visual observation (image) paired with structured textual annotations providing stage-dependent contextual prompts and monitoring point information.

**Output**: Binary anomaly detection label (normal or abnormal) per image.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)
```

## Common pitfalls

- Spatial and viewpoint diversity across fixed/mobile robotic arms may cause distribution shifts not captured in a single split.
- Consensus-based annotation by three annotators may smooth out rare or ambiguous anomalies, affecting ground-truth reliability.
- Zero-shot evaluation without task-specific fine-tuning may underestimate model capability on domain-specific scientific imagery.

## Evidence (verbatim from paper)

> demonstrating that higher-level contextual prompts significantly improve detection accuracy and robustness, validating the method's adaptability across diverse scientific processes.

## Citation

```bibtex
@misc{lin2025vlmanomaly,
  title={A VLM-based Method for Visual Anomaly Detection in Robotic Scientific Laboratories},
  author={Lin et al. (2025)},
  year={2025},
  note={arXiv:2506.05405}
}
```

- arXiv: 2506.05405