Saplma Truthfulness Eval

saplma-truthfulness-eval

The Internal State of an LLM Knows When It's Lying — Azaria et al. (2023) (arXiv:2304.13734, 2023)

What this evaluates

Evaluates whether an LLM's internal hidden layer activations can predict the veracity of a given statement. It probes the model's implicit knowledge of truthfulness by training a classifier on neural activations rather than relying on explicit prompting or output probabilities.

Datasets

• True-False Dataset — total ?; splits: train (-1), test (-1)
• LLM-Generated Statements — total 245; splits: train (-1), val (-1), test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • The proportion of correctly classified statements (true or false) out of the total number of evaluated statements.
• AUC — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve, measuring the model's discriminative ability across classification thresholds.

Input / output format

Input: A single factual statement (sentence).

Output: Binary classification label: 'true' or 'false'.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

Common pitfalls

• The evaluation uses a held-out validation set (30% of the generated statements) to determine the optimal classification threshold, rather than a fixed 0.5 cutoff.
• Baselines relying on LLM output probabilities are confounded by token frequency and sentence length, requiring ratio-based normalization or careful prompt design.

Evidence (verbatim from paper)

Table 1 and Figure 2 present the accuracy of all the models tested using the OPT-6.7b LLM, for each of the topics, along with the average accuracy. As depicted by the table and figure, SAPLMA clearly outperforms BERT and Few-shot learning, with BERT, 3-shot, and 5-shot learning achieving only slightly above a random guess (0.50).

Citation

@misc{azaria2023internal,
  title={The Internal State of an LLM Knows When It's Lying},
  author={Azaria et al. (2023)},
  year={2023},
  note={arXiv:2304.13734}
}

• arXiv: 2304.13734