Perplexity

perplexity

Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluation — Li et al. (2023) (arXiv:2309.10677, 2023)

What this evaluates

This protocol evaluates language model memorisation and training data contamination by measuring how well the model predicts benchmark text compared to out-of-distribution baselines. Lower perplexity on benchmark passages relative to a clean baseline indicates the model has likely seen the text during training.

Datasets

• QuAC — total ?; splits: test (-1)
• BoolQ — total ?; splits: test (-1)
• SQuAD_v2 — total ?; splits: test (-1)
• XSum — total ?; splits: test (-1)
• MMLU — total ?; splits: test (-1)

Metrics

• perplexity (primary) — range: other
  • Perplexity is computed as the exponentiated average negative log-likelihood of the tokens in the input text. The protocol compares the perplexity on benchmark samples against two baselines: a 'clean baseline' (text from post-training dates) and a 'memorised baseline' (text from training dates).

Input / output format

Input: Benchmark text samples (passages or documents), truncated to the average length of the respective benchmark.

Output: Perplexity score for the input text, compared against clean and memorised baseline perplexities.

Scoring recipe

def compute_contamination_metric(model, benchmark_text, clean_baseline_text, memorised_baseline_text):
    pplx_bench = compute_perplexity(model, benchmark_text)
    pplx_clean = compute_perplexity(model, clean_baseline_text)
    pplx_mem = compute_perplexity(model, memorised_baseline_text)
    # Lower perplexity on benchmark vs clean baseline indicates memorisation
    return pplx_clean - pplx_bench

Common pitfalls

• Truncating benchmark texts to average length may discard context critical for accurate perplexity estimation.
• Using Wikipedia/BBC news as baselines assumes uniform distribution across the domain, which may not hold for specific benchmark subsets.
• Perplexity is sensitive to tokenization and model version; results are not directly comparable across different LLM architectures without normalization.

Evidence (verbatim from paper)

We found all four foundation model exhibit significant memorisation behaviour on the three benchmarks. For large foundation models, e.g., gpt-3 and llama-13b,30b, benchmarks are largely memorised which leads to lower perplexity than the clean baseline.

Citation

@misc{li2023estimatingcontamination,
  title={Estimating Contamination via Perplexity: Quantifying Memorisation in Language Model Evaluation},
  author={Li et al. (2023)},
  year={2023},
  note={arXiv:2309.10677}
}

• arXiv: 2309.10677