Chinese Toxicity Detection Eval

chinese-toxicity-detection-eval

ToxiTrace: Gradient-Aligned Training for Explainable Chinese Toxicity Detection — Li et al. (2026) (arXiv:2604.12321, 2026)

What this evaluates

This protocol evaluates models on Chinese toxicity detection across two tasks: binary sentence-level classification and fine-grained toxic span extraction. It measures classification accuracy and precision/recall, while also assessing the model's ability to extract contiguous, human-readable toxic spans and the faithfulness of those explanations via confidence masking.

Datasets

• COLD — total 32480; splits: test (5323)
• ToxiCN — total 12011; splits: test (2411)
• CNTP — total 2533; splits: test (2533)

Metrics

• Accuracy (Acc) — range: [0, 1]
  • Proportion of correctly classified instances out of the total number of instances.
• Precision (P) — range: [0, 1]
  • Ratio of true positive predictions to all positive predictions (TP / (TP + FP)).
• Recall (R) — range: [0, 1]
  • Ratio of true positive predictions to all actual positive instances (TP / (TP + FN)).
• F1 (primary) — range: [0, 1]
  • Harmonic mean of Precision and Recall: 2 * (P * R) / (P + R).
• Macro-F1 — range: [0, 1]
  • Unweighted mean of F1 scores computed per class, treating all classes equally.
• Overlap F1 (primary) — range: [0, 1]
  • Span-level F1 where a prediction is correct if its overlap with the gold span exceeds 50%. Precision and Recall are computed over these binary correct/incorrect judgments.
• Character-level F1 — range: [0, 1]
  • Token-level F1 score computed by matching extracted character spans against ground-truth character spans.
• IoU — range: [0, 1]
  • Intersection over Union of character indices between predicted and gold spans.

Input / output format

Input: Chinese text sentences (for classification) or sentences with ground-truth toxic spans (for span extraction evaluation).

Output: Binary label (toxic/non-toxic) for classification; contiguous toxic span(s) or character-level attribution scores for extraction.

Scoring recipe

def compute_classification_metrics(preds, golds):
    tp = fp = fn = 0
    for p, g in zip(preds, golds):
        if p == 1 and g == 1: tp += 1
        elif p == 1 and g == 0: fp += 1
        elif p == 0 and g == 1: fn += 1
    acc = (tp + (len(preds) - tp - fp)) / len(preds)
    p = tp / (tp + fp) if (tp + fp) > 0 else 0
    r = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * p * r / (p + r) if (p + r) > 0 else 0
    return acc, p, r, f1

def compute_overlap_f1(pred_spans, gold_spans):
    correct = 0
    for p, g in zip(pred_spans, gold_spans):
        if len(g) == 0: continue
        overlap = len(set(p) & set(g)) / len(g)
        if overlap > 0.5: correct += 1
    return correct / len(gold_spans)

Common pitfalls

• Zero-shot LLMs perform poorly on Chinese toxicity tasks; fair comparison requires fine-tuning or instruction tuning, as noted in the experimental setup.
• Span extraction uses a strict >50% overlap threshold, which can penalize partial matches even if they capture the core toxic phrase.
• Character-level metrics in Chinese often yield fragmented spans; the protocol uses BiCSE to enforce contiguous span extraction for better evaluation.

Evidence (verbatim from paper)

To evaluate toxic content detection performance, we used five widely adopted metrics: Accuracy ($Acc$), Recall ($R$), Precision ($P$), $F_{1}$ and Macro-$F_{1}$ Score.

Citation

@misc{li2026toxitrace,
  title={ToxiTrace: Gradient-Aligned Training for Explainable Chinese Toxicity Detection},
  author={Li et al. (2026)},
  year={2026},
  note={arXiv:2604.12321}
}

• arXiv: 2604.12321