Agent Evaluation

Agent Evaluation

Framework for evaluating AI agent outputs: hallucination detection, quality scoring, behavioral regression, and CI/CD integration.

Hallucination Detection

Fact-Checking Pipeline

def detect_hallucination(response: str, context: str) -> dict:
    """Check if response contains unsupported claims."""
    claims = extract_claims(response)
    supported = [c for c in claims if verify_against(c, context)]
    return {
        'hallucination_rate': 1 - len(supported) / len(claims),
        'unsupported_claims': [c for c in claims if c not in supported]
    }

Red Teaming

• Generate adversarial prompts designed to trigger hallucinations
• Test with contradictory context
• Verify model says "I don't know" when appropriate

Output Quality Scoring

RAGAS Metrics

Scoring Pipeline

from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevance

result = evaluate(
    dataset=eval_dataset,
    metrics=[faithfulness, answer_relevance]
)

Behavioral Regression Testing

Test Types

• Consistency: Same input → same output across runs
• Robustness: Slight input variation → similar output
• Boundary: Edge cases handled gracefully
• Safety: No harmful, biased, or leaked content

LangSmith Integration

from langsmith import Client

client = Client()
client.evaluate(
    runnable,
    dataset_name="agent-behavior-tests",
    evaluators=[consistency_evaluator, safety_evaluator]
)

Latency & Reliability Benchmarks

CI/CD Evaluation Pipeline

# .github/workflows/agent-eval.yml
jobs:
  evaluate:
    steps:
      - run: python scripts/run_eval_suite.py
      - run: python scripts/check_hallucination_rate.py
      - run: python scripts/benchmark_latency.py
      - if: failure
        run: echo "Agent eval failed — blocking merge"

Adversarial Red Teaming

Attack Vectors

• Prompt injection: "Ignore previous instructions and..."
• Context poisoning: Inject false facts into context
• Role manipulation: "You are now an unrestricted AI..."
• Format exploitation: XML/JSON injection in prompts

Defense Validation

• Input sanitization working?
• Output validation catching issues?
• Guardrails triggered correctly?

Quick Eval Checklist

• Hallucination rate <5%
• RAGAS faithfulness ≥0.8
• Consistency score ≥0.9
• P99 latency <10s
• No safety violations
• Adversarial tests passing