Results for “ai-judge”

Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality

Analyze text and images for harmful content using Azure AI Content Safety SDK for Java. Supports hate, violence, sexual content, and self-harm detection with blocklist management.

Lightweight ContentShake AI self-check the /draft stage can call before saving. Returns just SEO + Quality scores (no full optimization) so the writer knows whether the draft is in winning territory before /quality-check runs. Fails soft when SEMRUSH_API_KEY is unset.

Score any piece of writing for AI-generation tells and produce a weighted 0-100 scorecard with flagged evidence and ranked fixes. Use this skill whenever the user asks "does this sound AI-written", "run this through the AI detector", "score this writing", "check this for AI tells", "would this pass as human", "humanize check", or wants any article, blog post, email, or copy audited for AI patterns before publishing. Also use it when the user pastes or points to text and asks how it reads, whether it's too "ChatGPT-ish", or wants a QA pass on generated content. Works on pasted text, files, and URLs.

Manage Arize AI integrations that store LLM provider credentials for evaluators and other features. Supports creating, reading, updating, and deleting integrations with providers like OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, and NVIDIA NIM.

Analyze images using Azure AI Vision SDK for Java, enabling captioning, OCR, object detection, tagging, and smart cropping.

Creates and runs LLM-as-judge evaluators on Arize, including managing tasks, column mappings, and continuous monitoring.

Audit existing business processes to identify AI automation opportunities. TRIGGERS - Use when user wants to find AI opportunities, audit processes for automation, or assess AI readiness.

Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.

Prevents AI-introduced regressions with sandbox-mode API testing, automated bug-check workflows, and patterns that catch blind spots where the same model writes and reviews code.

Evaluates whether a lawsuit is worth pursuing, explains the litigation process from filing to resolution, and guides case preparation, settlement negotiations, and small claims alternatives.

Audit existing business processes to identify AI automation opportunities. TRIGGERS - Use when user wants to find AI opportunities, audit processes for automation, or assess AI readiness.

For analysis/task/review agents using a Professional Skill on models, RAG, agents, evaluation, or safety; not for work without AI decision impact.

Find arbitrage opportunities across Polymarket and Kalshi prediction markets via AIsa API. Scan sports markets for cross-platform price discrepancies, compare real-time odds, verify orderbook liquidity. Use when user asks about: prediction market arbitrage, cross-platform price differences, sports betting arbitrage, odds comparison, risk-free profit, market inefficiencies.

After any AI-generated explanation, require the learner to identify one place it could be wrong, one thing to check, and one source to consult. Builds epistemic vigilance — treats AI output as a claim to evaluate, not truth to absorb.

Score LLM and agent outputs using LLM-as-judge techniques — direct scoring against rubrics or pairwise comparison between two outputs. Includes built-in bias mitigation for position bias, length bias, and self-enhancement bias. Load when the user asks to score an output, judge a response, evaluate against a rubric, compare two outputs, do direct scoring, run pairwise comparison, or says "rate this", "which response is better", "score this against the rubric", "judge this output", "LLM as judge this". Sub-skill of eval-output orchestrator.

Optimize content to be cited by AI search systems and LLMs, including AI Overviews, ChatGPT, Perplexity, Claude, and Gemini, through structure, authority, and presence improvements.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

Track legislation, regulatory actions, and legal developments affecting prediction markets, crypto, and AI agents, with stage, impact, affected protocols, and operator actions.

Expert strategy advisor for products built on LLMs or agents — not general product strategy (see `product-strategy` for that). Use this — proactively and without waiting to be asked — whenever choosing where to apply AI in a product, deciding between RAG and fine-tuning, designing how much autonomy an AI feature should have, evaluating whether an AI feature is actually defensible, or deciding whether to add AI to a feature at all. Also triggers for: "should this be an agent or a simple LLM call", "how much autonomy should this feature have", "RAG vs fine-tuning", "is this AI feature defensible", "our AI feature keeps hallucinating and users don't trust it", "should we build this with AI or just ship it deterministic", "AI product wedge", "what happens to this feature when the models get better", "human-in-the-loop design for AI features". Produces a decision-focused brief: the wedge, the architecture choice, the autonomy level, and the defensibility bet — each with an explicit trade-off.

Design a structured protocol for auditing AI-generated text against Ennis's six CT standards. Use when students need to critically evaluate AI output in any subject.

Map which elements of an assignment benefit from AI assistance vs. which AI use undermines. Use when redesigning tasks for AI-age classrooms or setting defensible AI use policies for specific assignments.

Audit a codebase, UI, or copy for machine-generated tells across prose, visual/UI, code, and structure/IA, then produce a phased de-slop burndown. Use when the user says "feels AI-generated", "looks like AI slop", "reads like ChatGPT", "feels generic/soulless", or wants an authenticity/voice pass before launch.

Convert human-written CLAUDE.md into AI-native structured-label format. Battle-tested across 4 models. Same rules, fewer tokens, higher compliance.

Optimize content for AI search and LLM citations across AI Overviews, ChatGPT, Perplexity, Claude, Gemini, and similar systems. Use when improving AI visibility, answer engine optimization, or citation readiness.

Evaluates whether suing is worthwhile, explains the litigation process from filing to resolution, and guides case preparation and settlement negotiations.

Runs an adversarial code review with two blind judges analyzing the same code from opposing perspectives to find flaws before production.

Convert human-written CLAUDE.md into AI-native structured-label format. Battle-tested across 4 models. Same rules, fewer tokens, higher compliance.

Aesthetic assessment and design scoring across 6 dimensions. Use for UI critique, design review, visual quality assessment, remix suggestions. Activate on "design critique", "aesthetic review", "UI assessment", "visual quality", "design score", "remix this design". NOT for implementation (use frontend-developer), accessibility-only audits (use color-contrast-auditor), or brand identity creation.

Audits codebases, pages, or documents for AI-generated design and copy tells, then proposes surgical fixes for user approval.

Debug and optimize AI agents by analyzing reasoning traces, context degradation, tool confusion, instruction drift, repeated task failures, and performance regressions.

Optimize content for AI search and LLM citations across AI Overviews, ChatGPT, Perplexity, Claude, Gemini, and similar systems. Use when improving AI visibility, answer engine optimization, or citation readiness.