Bias Detection Mitigation Eval

bias-detection-mitigation-eval

Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation — Görge et al. (2025) (arXiv:2512.10734, 2025)

What this evaluates

Evaluates a pipeline for detecting and mitigating representation bias and explicit stereotypes in text corpora. It measures how well the pipeline generates attribute-specific word lists, quantifies demographic representation imbalances, and identifies stereotypical language compared to human annotations and baselines.

Datasets

• Small Heap — total 50000000; splits: test (-1)
• Small Heap Neutral — total ?; splits: test (-1)
• StereoSet (filtered) — total 237; splits: test (237)
• Small Heap Annotated — total 134; splits: test (134)

Metrics

• DR score (primary) — range: [0, 0.5]
  • Quantifies representation bias by measuring deviation from a uniform distribution across sensitive attribute groups. Calculated per attribute; for gender, the maximum value is 0.5.
• Stereotype score — range: [0, 1]
  • Assesses the strength of explicit stereotypes in text on a continuous scale from 0 (small/no stereotype) to 1 (strong stereotype).

Input / output format

Input: Text snippets or sentences from corpora (e.g., Small Heap, StereoSet intersentence examples).

Output: For word list generation: lists of attribute-specific word tuples. For bias measurement: DR scores per attribute. For stereotype detection: a continuous score between 0 and 1 indicating stereotype strength.

Scoring recipe

def calc_dr(group_counts, total):
    expected = total / len(group_counts)
    return sum(abs(c - expected) for c in group_counts) / (2 * total)

def calc_overlap(generated, reference):
    return len(set(generated) & set(reference)) / len(reference)

def calc_stereotype(text, model):
    return model.score(text) # Returns float in [0, 1]

Common pitfalls

• The maximum possible DR score varies by sensitive attribute (e.g., 0.5 for gender), so scores must be interpreted relative to their attribute-specific bounds.
• StereoSet evaluation strictly excludes anti-stereotypes and only uses intersentence examples, which limits direct comparison to full StereoSet benchmarks.
• Frequency-based ranking for age/religion attributes yields ambiguous results; generation-based ranking is required instead.

Evidence (verbatim from paper)

We validate the $DR$ score as a bias indicator for representation biases by comparing $DR_{gender}$ scores between the Small Heap and Small Heap Neutral datasets.

Citation

@misc{gorg2025textualbias,
  title={Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation},
  author={Görge et al. (2025)},
  year={2025},
  note={arXiv:2512.10734}
}

• arXiv: 2512.10734