Se Toxicity Eval

se-toxicity-eval

A Benchmark Study of the Contemporary Toxicity Detectors on Software Engineering Interactions — Sarker et al. (2020) (arXiv:2009.09331, 2020)

What this evaluates

Evaluates the ability of contemporary toxicity detection models to correctly identify toxic language in software engineering contexts, such as code reviews and developer chat logs. It probes whether general-purpose classifiers can handle domain-specific terminology and contextual nuances without significant performance degradation.

Datasets

• Jigsaw Sample — total ?; splits: test (-1); repo https://github.com/WSU-SEAL/toxicity-dataset
• Code Review — total 6533; splits: test (-1); repo https://github.com/WSU-SEAL/toxicity-dataset
• Gitter Ethereum — total 4140; splits: test (-1); repo https://github.com/WSU-SEAL/toxicity-dataset

Metrics

• F-Score (primary) — range: [0, 1]
  • Harmonic mean of precision and recall: F1 = 2 * (Precision * Recall) / (Precision + Recall).
• Precision — range: [0, 1]
  • Ratio of correctly predicted toxic instances to all instances predicted as toxic.
• Recall — range: [0, 1]
  • Ratio of correctly predicted toxic instances to all actual toxic instances.
• Accuracy — range: [0, 1]
  • Ratio of correctly predicted instances (both toxic and non-toxic) to the total number of instances.
• Cohen's Kappa — range: [0, 1]
  • Measures inter-rater reliability between model predictions and human raters, adjusting for chance agreement.

Input / output format

Input: Raw text strings representing software engineering interactions (e.g., code review comments, Gitter chat messages).

Output: Binary toxicity label (toxic or non-toxic) per text instance.

Scoring recipe

def compute_metrics(predictions, gold_labels):
    tp = sum(1 for p, g in zip(predictions, gold_labels) if p == 1 and g == 1)
    fp = sum(1 for p, g in zip(predictions, gold_labels) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(predictions, gold_labels) if p == 0 and g == 1)
    tn = sum(1 for p, g in zip(predictions, gold_labels) if p == 0 and g == 0)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    accuracy = (tp + tn) / len(gold_labels)
    return {'precision': precision, 'recall': recall, 'f1': f1, 'accuracy': accuracy}

Common pitfalls

• Domain-specific words (e.g., 'kill', 'junk', 'execute') are frequently misclassified as toxic due to general-purpose training data.
• Self-deprecating or humble language common in SE culture (e.g., 'stupid question') triggers false positives.
• Different toxicity tools show low inter-agreement (Cohen's Kappa), meaning results vary significantly depending on the chosen detector.

Evidence (verbatim from paper)

By comparing each tools performance on the two SE datasets against its performance on the Jigsaw sample, we noticed significant degradations of F-scores. Both precisions and recalls of each tool dropped by more than 0.10 on the two SE datasets.

Citation

@misc{sarker2020toxicity,
  title={A Benchmark Study of the Contemporary Toxicity Detectors on Software Engineering Interactions},
  author={Sarker et al. (2020)},
  year={2020},
  note={arXiv:2009.09331}
}

• arXiv: 2009.09331