# Imdb Sentiment Eval

> Tests the model's capability to generate fixed-length representations for variable-length documents containing multiple sentences. It probes whether the method can scale to longer texts and outperform traditional bag-of-words baselines on a large-scale sentiment classification benchmark. Use when the user wants to benchmark on IMDB dataset, or asks about evaluating this task. Reports error rate.

- Skill: `qhjqhj00/imdb-sentiment-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/imdb-sentiment-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/imdb-sentiment-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/imdb-sentiment-eval

---


# imdb-sentiment-eval

> Distributed Representations of Sentences and Documents — Quoc V. Le et al. (2014) (arXiv:1405.4053, 2014)

## What this evaluates

Tests the model's capability to generate fixed-length representations for variable-length documents containing multiple sentences. It probes whether the method can scale to longer texts and outperform traditional bag-of-words baselines on a large-scale sentiment classification benchmark.

## Datasets

- **IMDB dataset** — total 100000; splits: train (25000), test (25000), unlabeled_train (50000); repo http://ai.Stanford.edu/amaas/data/sentiment/index.html

## Metrics

- `error rate` **(primary)** — range: percent
  - 1 - (number of correctly classified test instances / total test instances). Binary classification (Positive/Negative).

## Input / output format

**Input**: Movie reviews consisting of multiple sentences.

**Output**: Predicted sentiment label (Positive or Negative).

## Scoring recipe

```python
predictions = classifier.predict(paragraph_vectors)
errors = sum(1 for p, g in zip(predictions, gold_labels) if p != g)
error_rate = errors / len(gold_labels)
```

## Common pitfalls

- Paragraph vectors for test reviews are learned at test time via gradient descent after freezing the rest of the network.
- The model is trained on 75,000 documents (25k labeled + 50k unlabeled) to learn word and paragraph vectors before classification.
- A neural network with one hidden layer (50 units) is used instead of a linear logistic classifier.

## Evidence (verbatim from paper)

> The method described in this paper is the only approach that goes significantly beyond the barrier of 10% error rate. It achieves 7.42% which is another 1.3% absolute improvement (or 15% relative improvement) over the best previous result.

## Citation

```bibtex
@misc{le2014distributed,
  title={Distributed Representations of Sentences and Documents},
  author={Quoc V. Le et al. (2014)},
  year={2014},
  note={arXiv:1405.4053}
}
```

- arXiv: 1405.4053