# Content Safety Guardrail Evaluator

> Evaluate an AI system's content-safety guardrails against a supplied policy and approved test corpus. Use when a safety tester needs to measure unsafe acceptance, appropriate refusal or redirection, over-refusal, policy-category coverage, multi-turn behavior, obfuscation, or multilingual consistency before a release.

- Skill: `pramoddutta/content-safety-guardrail-evaluator` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add pramoddutta/content-safety-guardrail-evaluator`
- Raw SKILL.md: https://api.skillmd.com/api/skills/pramoddutta/content-safety-guardrail-evaluator/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: MIT
- Author: PramodDutta (https://skillmd.com/u/pramoddutta)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/pramoddutta/content-safety-guardrail-evaluator

---


# Content Safety Guardrail Evaluator

Measure policy behavior without inventing policy rules or treating automated evaluation as a
release decision. Balance unsafe acceptance with over-refusal of legitimate requests.

## Workflow

1. Confirm the policy owner, exact policy/taxonomy version, approved synthetic or sanitized
   corpus, authorized staging environment, reviewer access controls, and stop conditions.
2. Record the build, model/provider and version, system prompt revision, moderation/guardrail
   configuration, locale, temperature/seed when available, evaluator version, and run ID.
3. Map each case to a cited policy rule and expected allow, refuse, redirect, or escalate
   outcome. Mark uncited expectations `Not specified`; do not author policy by assumption.
4. Include approved positive controls, negative controls, benign near-boundary requests,
   multi-turn cases, transformations, obfuscation, and supported languages. Minimize exposure
   to harmful material and never expand it beyond the approved corpus.
5. Capture outputs and guardrail decisions. Report counts and denominators for unsafe
   acceptance, correct handling, over-refusal, inconsistent handling, and not-tested cases.
6. Record human reviewer disagreements and confidence limits where applicable. Present
   thresholds and release impact only as recommendations for the policy/release owners.

## Evidence register

Classify every statement as `Observed`, `Inferred`, `Hypothesis`, or `Not tested`.

| Classification | Source / run ID | Expected control | Observed result | Evidence artifact | Unresolved gap |
|---|---|---|---|---|---|
| Observed / Inferred / Hypothesis / Not tested | corpus item and repetition ID | cited policy rule | captured allow/refuse/redirect/escalate result | redacted output and decision trace | reviewer disagreement or missing category |

Record exact build, model, prompt, policy, guardrail, corpus, evaluator, locale, and environment
versions. Keep observed behavior separate from evaluator inference and root-cause hypotheses.

--- HUMAN REVIEW GATE ---
Unverified items:
Remaining risks:
Required approver:
Decision: approve / revise / reject

## Guardrails

- Run only in an explicitly authorized staging or sandbox environment using synthetic,
  sanitized, or otherwise approved inputs; restrict corpus and output access.
- Never fabricate a policy rule, expected label, model output, evaluation result, denominator,
  metric, reviewer decision, source, trace, or evidence link.
- Never claim safe, compliant, certified, unbiased, or fully covered from an automated or
  incomplete evaluation. A threshold result is evidence, not a release decision.
- Pause and escalate content outside the approved corpus or reviewer handling rules; require
  the policy and release owners to approve conclusions and deployment impact.