ncreighton AI Refund Request Analyzer & Policy Loophole Identifier
Overview
The AI Refund Request Analyzer is a production-grade risk management tool designed for solopreneurs, agencies, and small-to-medium businesses to intelligently process refund requests while protecting profit margins and maintaining customer goodwill.
This skill combines machine learning pattern recognition with policy analysis to:
- Detect fraud signals — Identifies serial refunders, suspicious timing patterns, and coordinated abuse
- Recommend decisions — Provides approve/deny/counter-offer recommendations with confidence scores
- Learn from history — Builds predictive models from your historical refund data to improve accuracy over time
- Flag policy gaps — Suggests refund policy improvements based on edge cases and loopholes
- Integrate seamlessly — Works with Stripe, Shopify, WooCommerce, WordPress, Slack, and Google Sheets for end-to-end automation
Why it matters: Manual refund review wastes 3-5 hours per week for growing businesses. This skill automates 70-80% of decisions while flagging high-risk cases for human review, reducing fraud losses by 40-60% without damaging customer relationships.
Quick Start
Example 1: Analyze a Single Refund Request
Analyze this refund request for fraud risk:
- Customer: john.smith.2847@gmail.com
- Order ID: #ORD-2024-18473
- Amount: $249.00
- Product: "Advanced SEO Course Bundle"
- Purchase date: 2024-01-15
- Refund request date: 2024-01-18 (3 days after purchase)
- Reason: "Not what I expected"
- Customer account age: 2 months
- Previous purchases: 1 (similar product, refunded 45 days ago)
- Previous refunds: 2 (both within 7 days of purchase)
- Chargeback history: None
- Device fingerprint matches: 3 other accounts
Provide:
1. Fraud risk score (0-100)
2. Decision recommendation (approve/deny/counter-offer)
3. Key risk factors
4. Suggested response message
Result: The skill returns a risk assessment with confidence scores, identifies the customer as a likely "serial refunder," and suggests a counter-offer (store credit instead of cash refund).
Example 2: Bulk Analyze Refund Requests from CSV
Analyze these 15 refund requests from my Stripe account for patterns:
Customer Email,Order ID,Amount,Days Since Purchase,Reason,Previous Refunds,Account Age
sarah.j@email.com,#ORD-2024-18401,$199.00,5,"Changed mind",0,180
mike.t@email.com,#ORD-2024-18402,$89.99,2,"Not satisfied",3,45
lisa.m@email.com,#ORD-2024-18403,$349.00,14,"Quality issue",0,365
david.k@email.com,#ORD-2024-18404,$149.00,1,"Wrong item",0,10
...
Identify:
1. Fraud rings or coordinated abuse
2. High-risk refunders (serial abusers)
3. Legitimate complaints
4. Policy gaps being exploited
5. Recommended actions for each request
Result: Bulk analysis with clustering of similar patterns, risk tier assignments, and automated Slack notifications for high-priority cases.
Example 3: Identify Policy Loopholes & Generate Improvements
Analyze our current refund policy for exploitable loopholes:
Current policy: "30-day money-back guarantee on all digital products.
No questions asked. Refunds processed to original payment method within 5 business days."
Historical data: 847 refund requests over 12 months
- Approval rate: 92%
- Fraud-suspected cases: 23 (2.7%)
- Average refund amount: $156.00
- Estimated loss to abuse: $8,400/year
Identify loopholes and suggest policy improvements that:
1. Close abuse vectors
2. Maintain customer satisfaction (target: 85%+ approval for legitimate claims)
3. Add friction only for high-risk scenarios
4. Are legally compliant
Result: Detailed policy audit with specific loopholes flagged (e.g., "30-day window allows course completion before refund"), improvement recommendations, and A/B testing suggestions.
Capabilities
1. Fraud Pattern Detection
The skill analyzes refund requests against 25+ fraud indicators:
- Serial refunder detection — Flags customers with 3+ refunds in 12 months or 2+ refunds on similar products
- Timing anomalies — Identifies suspicious patterns (refunds within 24 hours, clustered on weekends, after promotional emails)
- Device/IP clustering — Detects multiple accounts from same device, IP range, or email domain variations
- Behavioral scoring — Compares against your historical baseline (e.g., if 5% of customers refund, a customer with 40% refund rate is flagged)
- Chargeback correlation — Links refund requests to previous chargebacks or payment disputes
- Content analysis — Scans refund reason text for generic/copy-paste language suggesting coordinated abuse
Usage:
Analyze this refund for fraud signals:
Customer: jane@example.com | Order: $299 course | 4 days post-purchase
Previous: 5 refunds in 6 months on digital products
Device: Matches 2 other high-refund-rate accounts
Reason text: "Not as described" (generic language)
Flag: HIGH RISK (serial refunder + device clustering + generic reason)
Confidence: 87%
2. Decision Recommendation Engine
Generates approve/deny/counter-offer recommendations with explainable reasoning:
- Approve — Legitimate claim with low fraud risk. Suggested message emphasizes customer satisfaction.
- Deny — High fraud signals or policy violation. Suggested message with legal/policy justification.
- Counter-offer — Medium risk or partial claim. Suggests store credit, partial refund, or replacement.
- Escalate to human — Ambiguous cases requiring judgment (e.g., quality disputes, mixed signals).
Example output:
DECISION: Counter-offer
CONFIDENCE: 76%
REASONING: Customer has 2 previous refunds (policy threshold is 3).
Account age is 6 weeks (below 90-day threshold for auto-approve).
Reason is legitimate ("quality issue") but timing is suspicious (11 days).
RECOMMENDATION: Offer 50% refund + store credit for $75 toward replacement product.
This retains $150 while showing good faith.
CUSTOMER MESSAGE:
"Thank you for reaching out. We're sorry the product didn't meet expectations.
To make this right, we'd like to offer you $75 in store credit toward any
product in our catalog, plus a 50% refund ($124.50). This lets you try
something else risk-free. Would that work for you?"
3. Historical Learning & Predictive Modeling
Builds custom ML models from your refund history:
- Imports historical data — Connects to Stripe, Shopify, WooCommerce, or Google Sheets
- Trains models — Learns which customers/patterns result in chargebacks, disputes, or re-refunds
- Improves over time — Feedback loop: you mark decisions as correct/incorrect, model accuracy increases
- Benchmarking — Compares your refund rate, fraud rate, and policy to industry standards
Monthly model retraining ensures the skill adapts to your business changes (new products, customer base shifts, seasonal patterns).
4. Policy Gap Analysis & Recommendations
Audits your refund policy against your historical data:
- Identifies exploited loopholes — Finds edge cases where policy wording allows abuse
- Suggests improvements — Proposes specific policy changes with estimated impact on fraud reduction
- A/B test recommendations — Suggests policy changes to test (e.g., "Require video proof of product issue for claims >$200")
- Compliance check — Ensures recommendations comply with FTC, GDPR, and payment processor rules
Example:
LOOPHOLE: "30-day money-back guarantee" + "No questions asked" =
Customers can complete digital courses (1-2 weeks of access) and refund.
Estimated abuse: 12-15 cases/month × $150 = $1,800-2,250/month loss.
IMPROVEMENT: Change to "30-day satisfaction guarantee. Digital products
are non-refundable after first access. If you're unsatisfied before
accessing, full refund available."
IMPACT: Reduces abuse by ~70% (industry benchmark: 65-75%).
Maintains 90%+ satisfaction on legitimate claims.
5. Slack & Email Integration
Automates notifications and workflows:
- Real-time alerts — Sends Slack message for high-risk refunds (>80 fraud score)
- Bulk reports — Daily/weekly summaries of refund trends, top risk factors, and policy recommendations
- One-click actions — Slack buttons to approve/deny/counter-offer directly (updates Stripe/Shopify automatically)
- Customer outreach — Generates and sends suggested response messages via email
Example Slack message:
🚨 HIGH-RISK REFUND DETECTED
Customer: john@example.com | Order #ORD-2024-18473 | $249.00
Fraud Score: 87/100 | Serial Refunder (4 refunds in 6 months)
Reason: "Not what I expected"
Device: Matches 3 other high-risk accounts
RECOMMENDED ACTION: Deny + Offer store credit
[Approve] [Deny] [Counter-Offer] [Escalate]
Configuration
Required Environment Variables
# OpenAI API for pattern analysis and policy recommendations
export OPENAI_API_KEY="sk-..."
# Stripe for payment/refund data (if using Stripe)
export STRIPE_API_KEY="sk_live_..."
export STRIPE_SECRET_KEY="rk_live_..."
# Slack for notifications (optional but recommended)
export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/T00000000/B00000000/..."
# Google Sheets for historical data import (optional)
export GOOGLE_SHEETS_API_KEY="..."
export GOOGLE_SHEET_ID="..."
Setup Instructions
Step 1: Connect Your Data Source
# Option A: Stripe (recommended for e-commerce)
claw config set stripe_mode live
claw config set stripe_account_id acct_xxxxx
# Option B: CSV Upload
claw refund-analyzer import-csv refund_history.csv
# Option C: Google Sheets
claw refund-analyzer connect-sheets https://docs.google.com/spreadsheets/d/xxxxx
Step 2: Set Your Policy Baseline
Configure your refund policy:
- Maximum refund window: 30 days
- Refund approval threshold: 90% (approve 90% of claims by default)
- Serial refunder threshold: 3 refunds in 12 months
- High-risk amount threshold: $500+
- Escalation email: compliance@yourcompany.com
Step 3: Enable Integrations
# Enable Slack notifications
claw refund-analyzer enable-slack
# Enable automatic Stripe refund processing (optional)
claw refund-analyzer enable-stripe-automation --approval-only
# Enable daily reports
claw refund-analyzer schedule-report daily 9am
Example Outputs
Single Refund Analysis Report
{
"request_id": "REF-2024-001847",
"customer_email": "john.smith@example.com",
"order_id": "ORD-2024-18473",
"refund_amount": 249.00,
"fraud_risk_score": 78,
"fraud_risk_level": "HIGH",
"recommendation": "COUNTER-OFFER",
"confidence": 0.82,
"decision_reasoning": [
"Serial refunder: 4 refunds in 6 months (threshold: 3)",
"Timing anomaly: Refund requested 3 days post-purchase (baseline: 8 days)",
"Device clustering: Matches 3 other high-risk accounts",
"Generic reason text: 'Not what I expected' (low specificity score: 0.34)"
],
"risk_factors": {
"account_age_days": 60,
"previous_refunds": 4,
"days_since_purchase": 3,
"chargeback_history": 0,
"device_match_count": 3,
"reason_specificity": 0.34
},
"suggested_action": {
"type": "counter_offer",
"offer_details": "50% refund ($124.50) + $75 store credit",
"rationale": "Retains revenue while showing good faith. Reduces abuse incentive."
},
"suggested_message": "Thank you for reaching out. We're sorry the product didn't meet expectations. To make this right, we'd like to offer you $75 in store credit toward any product in our catalog, plus a 50% refund ($124.50). This lets you try something else risk-free. Would that work for you?",
"next_steps": [
"Send counter-offer message",
"Monitor for response within 48 hours",
"If accepted, process refund automatically",
"If declined, escalate to human review"
]
}
Bulk Analysis Report (15 Refund Requests)
REFUND ANALYSIS SUMMARY (2024-01-15 to 2024-01-18)
================================================
Total Requests: 15
Processed: 15
High Risk: 3
Medium Risk: 5
Low Risk: 7
RECOMMENDATIONS:
- Approve: 7 (46.7%)
- Counter-offer: 5 (33.3%)
- Deny: 2 (13.3%)
- Escalate: 1 (6.7%)
FRAUD SIGNALS DETECTED:
1. Serial Refunder Ring: 3 accounts (john@email.com, jane@email.com,
mike@email.com) with coordinated refund requests within 6-hour window.
Estimated loss if approved: $627.00
Recommendation: DENY all three + flag for review
2. Device Clustering: 4 accounts from same IP range (192.168.1.x) with
similar refund patterns. Likely coordinated abuse.
Recommendation: Manual review + potential account ban
3. Policy Loophole: 8 of 15 requests exploit "30-day window" by refunding
after completing digital product access. Estimated recurring loss:
$1,200-1,500/month.
Recommendation: Update policy language (see below)
POLICY IMPROVEMENTS RECOMMENDED:
1. Change "No questions asked" to "No questions asked before first access"
2. Add: "Digital products are non-refundable after first access"
3. Add: "Refunds for quality issues require photo/video evidence"
4. Implement: 2-factor verification for refund requests >$200
ESTIMATED IMPACT:
- Fraud reduction: 60-70%
- Legitimate claim approval rate: 88-92% (current: 92%)
- Monthly savings: $800-1,200
Policy Audit Report
REFUND POLICY AUDIT
===================
Current Policy Score: 6.2/10 (moderate risk)
LOOPHOLES IDENTIFIED:
1.