AI Community Toxic Comment Classifier

AI Community Toxic Comment Classifier & Response Recommendation Engine

Overview

This production-ready skill automates community moderation by analyzing user comments, classifying toxicity types, calculating severity scores (0-100), and recommending tailored response strategies. Unlike basic keyword filters, this AI-powered system understands context, intent, and nuance—distinguishing between spam, hate speech, misinformation, bad faith criticism, and legitimate concerns.

Why it matters: Community managers spend 15-20 hours/week on moderation. This skill reduces that to minutes while improving consistency and response quality. It integrates seamlessly with WordPress (WP REST API), Discord (Bot API), Slack (incoming webhooks), Google Sheets (append-only audit logs), and custom platforms via JSON webhooks.

Key value propositions:

• Reduces moderation response time by 85%
• Provides consistent, templated responses aligned with your community values
• Creates audit trails for compliance (GDPR, platform policies)
• Learns from your specific community guidelines over time
• Prevents escalation through intelligent triage (delete vs. respond vs. DM)

Quick Start

Try these prompts immediately to see the skill in action:

Example 1: Basic Toxicity Analysis

Analyze this comment for toxicity:
"Your product is garbage and anyone who buys it is an idiot. 
This company is a scam and should be shut down."

Expected output: Severity 78/100, Type: Hate Speech + Bad Faith Criticism, Recommended action: Respond publicly with empathy template, offer refund/support.

Example 2: Batch Processing with Slack Integration

Process these 5 comments and send moderation summary to Slack webhook:

1. "This feature request is stupid lol"
2. "I love your product! Just a small bug with the API docs"
3. "BUY CRYPTO NOW!!! CLICK LINK!!!"
4. "Your competitor's product is better because of X, Y, Z"
5. "Kill all [group]. This product enables genocide."

Webhook URL: https://hooks.slack.com/services/YOUR/WEBHOOK/URL
Channel: #moderation-queue

Example 3: WordPress Integration with Audit Trail

Fetch pending comments from WordPress site:
URL: https://mysite.com
API credentials: (use env var WORDPRESS_API_TOKEN)

Classify each comment. For severity >= 70:
- Flag for manual review
- Log to Google Sheet: https://docs.google.com/spreadsheets/d/SHEET_ID
- Send DM template to comment author
- Recommend deletion with reason code

Example 4: Learning from Community Guidelines

Upload my community guidelines PDF and learn response patterns:
File: community-guidelines.pdf

Then classify this comment using those learned patterns:
"This is off-topic but I wanted to share my political views..."

Capabilities

1. Multi-Type Toxicity Classification

The skill categorizes comments into these types (multiple can apply):

• Spam: Promotional links, bot-generated content, repetitive messages
• Hate Speech: Dehumanizing language, slurs, calls for violence against groups
• Misinformation: False factual claims, conspiracy theories, dangerous health advice
• Bad Faith Criticism: Attacks on character vs. ideas, strawmanning, bad-faith arguments
• Legitimate Criticism: Valid concerns, feature requests, bug reports (marked as non-toxic)
• Off-Topic: Content outside community scope (low priority)
• Sarcasm/Dark Humor: Context-aware detection to avoid false positives

Usage example:

Classify with detailed reasoning:
"Vaccines cause autism and the government is hiding the truth!"

Include:
- Primary type
- Secondary types (if any)
- Confidence score per type
- Specific phrases triggering classification

2. Severity Scoring (0-100 Scale)

Dynamic scoring based on:

• Toxicity intensity (language, tone, intent)
• Community impact (targeting vulnerable groups, encouraging action)
• Context (first offense vs. repeat offender)
• Scope (individual attack vs. systemic claim)

Example:

Score severity for:
"You're wrong about X" → 5/100 (legitimate disagreement)
"You're an idiot" → 25/100 (personal attack, low impact)
"[Slur] should be killed" → 95/100 (hate speech, calls for violence)

3. Intelligent Response Recommendation Engine

Suggests actions with templated responses:

Example output:

Action: RESPOND_PUBLICLY
Template: "Thanks for your feedback. We understand your frustration. 
Here's how we address [concern]. If you'd like to discuss further, 
please DM us or email support@..."

Alternative templates:
- Empathetic (acknowledge emotion first)
- Firm (clear boundary)
- Educational (provide resources)
- Redirect (move to appropriate channel)

4. Integration Connectors

• WordPress: Fetch pending comments, auto-approve/trash, reply via REST API
• Discord: Analyze messages, flag in mod-queue channel, soft-delete with DM
• Slack: Send moderation summaries, approve/reject from thread
• Google Sheets: Append audit logs with timestamp, comment text, classification, action taken
• Webhooks: POST classification results to custom platforms (Discourse, Mighty Networks, etc.)

5. Adaptive Learning

• Upload community guidelines, values statements, or past moderation decisions
• Skill learns your specific tone, thresholds, and response patterns
• Improves classification accuracy over time (within single session)
• Generates "moderation playbook" recommendations

Usage:

Learn from my moderation history:
- Google Sheet with 200 past decisions: https://docs.google.com/spreadsheets/d/ID
- Community guidelines PDF
- Tone guide (formal vs. casual)

Then re-classify these 10 comments using learned patterns.

6. Audit Trail & Compliance

• Logs every classification decision with reasoning
• Exports compliance-ready reports (GDPR, platform policy adherence)
• Timestamps all actions
• Tracks decision consistency over time

Configuration

Environment Variables (Required)

# OpenAI API for classification intelligence
OPENAI_API_KEY=sk-...

# Optional: Slack webhook for real-time moderation alerts
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...

# Optional: WordPress REST API credentials
WORDPRESS_API_TOKEN=your_token_here
WORDPRESS_SITE_URL=https://yoursite.com

# Optional: Google Sheets API for audit logs
GOOGLE_SHEETS_API_KEY=your_key_here
GOOGLE_SHEET_ID=sheet_id_here

Skill Options

--severity-threshold=50        # Only flag comments above this score
--batch-size=20                # Process up to 20 comments per run
--language=en                  # Supports: en, es, fr, de, zh, ja
--learning-mode=true           # Enable adaptive learning from guidelines
--audit-log=true               # Enable Google Sheets logging
--dry-run=false                # Set true to preview without taking action
--response-template-set=professional  # Options: professional, casual, firm, empathetic

Setup Steps

1. Get API Keys:

   • OpenAI: https://platform.openai.com/api-keys
   • Slack: Create incoming webhook in workspace settings
   • WordPress: Generate application password in user settings
   • Google Sheets: Enable API and create service account

2. Set Environment Variables:

   export OPENAI_API_KEY=sk-...
   export SLACK_WEBHOOK_URL=https://hooks.slack.com/...
   

3. Test Integration:

   Test WordPress connection:
   site_url: https://mysite.com
   api_token: (from env)
   Expected: Returns 5 pending comments
   

4. (Optional) Upload Community Guidelines:

   • PDF, Google Doc, or plain text
   • Skill extracts values and moderation patterns
   • Uses for context in future classifications

Example Outputs

Single Comment Classification

{
  "comment_id": "12345",
  "text": "This product is terrible and you should all switch to [competitor]",
  "author": "john_doe",
  "timestamp": "2024-01-15T14:32:00Z",
  "classification": {
    "primary_type": "bad_faith_criticism",
    "secondary_types": ["legitimate_criticism"],
    "severity_score": 42,
    "confidence": 0.89,
    "reasoning": "Contains legitimate product comparison but frames it as absolute judgment without constructive feedback"
  },
  "recommended_action": "RESPOND_PUBLICLY",
  "response_template": "empathy_plus_redirect",
  "suggested_response": "We appreciate you sharing your experience! We'd love to understand what specific features are missing for you. Can you tell us more about your use case? Our support team at support@... can also discuss alternatives.",
  "alternative_actions": ["APPROVE_AND_HIGHLIGHT", "DM_PRIVATELY"],
  "learning_notes": "Similar to past decision #89. User shows pattern of comparison criticism. Consider monitoring for repeat offense."
}

Batch Processing Summary

Moderation Report: 2024-01-15
Total comments processed: 47
Distribution:
  Approved (0-20): 28 comments
  Monitor (21-50): 12 comments
  Review (51-75): 5 comments
  Delete (76-100): 2 comments

High-priority items:
  - Comment #445 (Severity 89): Hate speech + calls for violence
    Action: DELETE + DM author with resources
  - Comment #312 (Severity 76): Misinformation about safety
    Action: RESPOND_PUBLICLY + add fact-check link

Patterns detected:
  - 3 spam comments from same IP (recommend IP ban)
  - Misinformation cluster around COVID (10 comments)
  - Legitimate feature requests (8 comments, highlight for product team)

Audit trail: https://docs.google.com/spreadsheets/d/SHEET_ID
Slack summary: Posted to #moderation-queue

Slack Integration Output

🛡️ Moderation Alert (2024-01-15 14:35 UTC)

⚠️ HIGH PRIORITY (Severity 87/100)
Comment ID: #445 | Author: @toxic_user
Type: Hate Speech + Calls for Violence
Text: "[Slur] should be... [REDACTED]"
Recommended: DELETE + DM with resources
Action buttons: [Delete] [Respond] [Defer to Mod] [Learn from this]

📊 Daily Summary
47 comments | 28 approved | 12 flagged | 5 under review | 2 deleted
Consistency score: 94% (vs. past moderation)

Tips & Best Practices

1. Start with Dry-Run Mode

Run with --dry-run=true for first 100 comments
Review recommendations before enabling auto-actions
Adjust severity thresholds based on false positive rate

2. Upload Community Guidelines Early

• Skill learns your specific values and tone
• Classification accuracy improves 15-25% with guidelines
• Update guidelines quarterly as community evolves
• Include examples of approved vs. rejected comments

3. Use Response Templates Strategically

• Professional tone: B2B communities, formal support
• Casual tone: Gaming, hobby communities, Discord
• Firm tone: Zero-tolerance policies, safety-critical
• Empathetic tone: Mental health, support communities
• Mix templates: Use firm + empathetic for serious violations

4. Monitor False Positives Weekly

Check audit log for:
- Sarcasm flagged as hate speech
- Legitimate criticism marked as bad faith
- Off-topic marked as spam (when relevant)

Adjust learning model based on 5+ false positives of same type

5. Integrate with Moderation Workflow

• Severity 76-100: Auto-delete (with audit trail)
• Severity 51-75: Send to #moderation-queue in Slack (human review)
• Severity 21-50: Auto-respond with template (human can edit before sending)
• Severity 0-20: Auto-approve (human can override)

6. Track Repeat Offenders

• Skill identifies users with 3+ violations
• Recommend escalating action (DM warning → temporary mute → ban)
• Export repeat offender list monthly for pattern analysis

7. A/B Test Response Templates

• Track engagement rate on different template types
• Monitor if firm responses reduce repeat violations
• Adjust based on community response

Safety & Guardrails

What This Skill Will NOT Do

• Not a legal tool: Does not replace legal review for compliance (GDPR, CCPA, etc.). Always consult legal counsel.
• Not a judge: Recommendations are suggestions, not final decisions. Humans retain full authority.
• Not bias-free: AI models reflect training data biases. Regularly audit for disparate impact across user groups.
• Not a surveillance tool: Should not be used to monitor employees, private messages, or non-public content without consent.
• Not for harassment: Cannot be weaponized to target individuals or groups. Skill will refuse requests that appear retaliatory.

Boundaries & Limitations

1. Context loss: May misclassify sarcasm, idioms, or cultural references (especially non-English)
2. Evolving slurs: Cannot catch novel slurs or coded language (requires human review for edge cases)
3. False positives on satire: May flag legitimate satire as hate speech (review before deletion)
4. No private data learning: Skill does not learn from deleted comments or private messages
5. Rate limits: OpenAI API has rate limits (100 requests/min on free tier; higher on paid)
6. Language support: Best accuracy in English; moderate accuracy in Spanish, French, German; lower in others

Ethical Guidelines

• Transparency: Inform community members that AI moderation is in use
• Appeal process: Always allow users to appeal classifications (provide support email)
• No shadow banning: Don't use skill to silently suppress comments without user knowledge
• Diverse moderation team: Don't rely solely on AI; pair with human moderators from diverse backgrounds
• Regular audits: Monthly review of classification patterns for bias (by demographic, topic, etc.)

Compliance Notes

• GDPR: Audit logs include personal data; ensure retention policy complies (delete after 90 days if not needed)
• Platform policies: Verify skill's actions comply with WordPress, Discord, Slack ToS
• Defamation risk: Ensure response templates don't make false claims about users
• Accessibility: Ensure Slack/email notifications are accessible (alt text for images, captions for videos)

Troubleshooting

Common Issues & Solutions

Q: Skill is classifying legitimate criticism as "bad faith"

• A: Upload community guidelines with examples of good vs. bad criticism. Adjust --severity-threshold lower. Review the "reasoning" field in output to understand the decision.

Q: False positives on sarcasm (e.g., "Great job breaking the site again!")

• A: Sarcasm is context-dependent. Skill has lower confidence on sarcasm (check confidence field). For communities heavy in sarcasm, use --response-template-set=casual and manually