Civil Comments Toxicity Eval

civil-comments-toxicity-eval

Beyond Accuracy: An Explainability-Driven Analysis of Harmful Content Detection — Dhara, Siddhesh Sheth (2026) (arXiv:2603.18015, 2026)

What this evaluates

Evaluates a RoBERTa-based classifier's ability to detect toxic or harmful content in online comments. It probes the model's sensitivity to explicit lexical cues versus implicit, context-dependent toxicity, highlighting failure modes that aggregate accuracy metrics miss.

Datasets

• Civil Comments — total 4000; splits: test (4000)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Fraction of correctly classified instances out of the total test set.
• AUC — range: [0, 1]
  • Area under the receiver operating characteristic curve, measuring the model's ability to distinguish between classes across all classification thresholds.
• Precision (Toxic) — range: [0, 1]
  • Ratio of true positive toxic predictions to all predicted toxic instances.
• Recall (Toxic) — range: [0, 1]
  • Ratio of true positive toxic predictions to all actual toxic instances.
• F1-score (Toxic) — range: [0, 1]
  • Harmonic mean of Precision (Toxic) and Recall (Toxic), balancing false positives and false negatives for the minority toxic class.

Input / output format

Input: Raw text of online comments.

Output: Binary label: 'toxic' or 'non-toxic' (neutral).

Scoring recipe

def compute_metrics(y_true, y_pred):
    tp = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 1)
    fp = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 1)
    fn = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 0)
    tn = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 0)
    accuracy = (tp + tn) / len(y_true)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    return {'accuracy': accuracy, 'precision': precision, 'recall': recall, 'f1': f1}

Common pitfalls

• Relying solely on accuracy masks poor performance on the minority toxic class due to severe class imbalance.
• Surface-level lexical cues can trigger false positives on politically affiliated or emotionally expressive but non-toxic comments.
• Implicit, rhetorical, or context-dependent toxicity is frequently missed (false negatives) because the model over-relies on explicit keywords.

Evidence (verbatim from paper)

Table 1 summarizes the classification results using accuracy, precision, recall, and F1-score for the toxic class, along with the area under the receiver operating characteristic curve. The model correctly predicts 3,566 examples out of the 4,000 test samples.

Citation

@misc{dhara2026beyondaccuracy,
  title={Beyond Accuracy: An Explainability-Driven Analysis of Harmful Content Detection},
  author={Dhara, Siddhesh Sheth (2026)},
  year={2026},
  note={arXiv:2603.18015}
}

• arXiv: 2603.18015