manastalukdar LLM QA — Output Quality Investigation
Root-cause analysis for LLM output failures: hallucinations, refusals, format errors, incomplete answers, and regression in answer quality.
Usage
/llm-qa "<description of the quality issue>" # investigate a specific failure
/llm-qa --regression # compare output quality before/after a change
/llm-qa --prompts <path> # audit prompt files for known quality anti-patterns
Behavior
Step 1 — Classify the failure type
Identify which category the quality issue falls into:
| Failure type | Symptoms | Primary suspects |
|---|---|---|
| Hallucination | Confident wrong facts, invented citations | Context gap, stale retrieval, no grounding |
| Refusal | Unexpected "I can't help with that" | System prompt over-restriction, safety over-triggering |
| Format error | Malformed JSON, missing fields, wrong structure | No output schema, inconsistent instructions |
| Incomplete answer | Truncated, missing steps, dropped context | max_tokens too low, context window overflow |
| Quality regression | Correct before, wrong after recent change | Prompt changed, model changed, retrieval changed |
| Inconsistency | Different answers to the same question | Temperature too high, no seed, non-deterministic retrieval |
Step 2 — Reconstruct the failing call
Ask for (or locate in code):
- The exact system prompt used
- The user input that triggered the failure
- The retrieved context (if RAG)
- The model and parameters (temperature, max_tokens, top_p)
- The actual vs expected output
# Find where the failing call is constructed
grep -rn "system_prompt\|systemPrompt\|messages\s*=" --include="*.py" --include="*.ts" -n . | head -20
Step 3 — Audit context quality
Check whether the model had the information it needed:
- Context gap: Was the correct information present in the retrieved chunks or conversation history?
- Context relevance: Were irrelevant chunks crowding out relevant ones? Check similarity scores.
- Context recency: Is the retrieved information current? Check timestamps or version markers.
- Context length: Was the context near the model's limit? Truncation causes hallucination at boundaries.
# Estimate tokens in context (rough: 1 token ≈ 4 chars)
wc -c context.txt | awk '{print int($1/4), "approx tokens"}'
Step 4 — Audit the prompt
Check the system prompt and user message for known quality failure patterns:
| Anti-pattern | Example | Fix |
|---|---|---|
| Contradictory instructions | "Be concise" + "explain in detail" | Remove one; make priority explicit |
| Ambiguous output format | "Return JSON" without schema | Provide exact JSON schema or use structured output |
| Missing grounding instruction | No "only use provided context" | Add explicit grounding constraint |
| Persona over-restriction | "Never discuss X" too broad | Narrow to specific harmful case |
| Missing failure instruction | No "if unsure, say so" | Add explicit uncertainty handling |
| Long system prompt noise | 2,000+ token system prompt with irrelevant rules | Trim to minimum necessary |
Step 5 — Audit model parameters
| Parameter | Risk if wrong | Check |
|---|---|---|
temperature |
> 0.7 increases inconsistency | Set 0–0.3 for factual tasks |
max_tokens |
Too low truncates response | Check against expected output length |
top_p |
Combined with high temp amplifies randomness | Use one of temp or top_p, not both |
seed |
Absent means non-deterministic | Set for reproducibility in tests |
Step 6 — Identify root cause and fix
Map the failure to its root cause layer:
Root cause layers (in order of likelihood):
1. Context gap / retrieval miss → fix retrieval query or chunk strategy
2. Prompt instruction conflict → remove or clarify conflicting rule
3. Missing output schema → add structured output / response format
4. Context overflow → reduce chunk count or trim system prompt
5. Model parameter → adjust temperature or max_tokens
6. Model capability limit → switch to a more capable model for this task
Step 7 — Propose regression test
For each root cause fixed, generate a test case to prevent recurrence:
# Regression test template
def test_llm_no_hallucination_on_unknown_topic():
response = llm_call(
system="Only answer based on the provided context. If unsure, say 'I don't know'.",
user="What is the capital of Zorbania?",
context="" # empty context — model must refuse, not hallucinate
)
assert "don't know" in response.lower() or "not sure" in response.lower()
Edge Cases
- No access to logs or prompt: Guide the user to capture the failing call; provide a logging snippet to add.
- Regression after model upgrade: Check for model-specific behavior changes; test with previous model version if possible.
- Non-English output failure: Check tokenization differences for the target language; consider language-specific prompts.
- Tool call failures in agentic loop: Trace the tool result that re-entered the context and check if it introduced confusion.
Token Optimization
Expected range: 400–1,500 tokens (full investigation); 200–500 tokens (--prompts audit mode)
Patterns used: Grep-before-Read (locate prompt files before reading), progressive disclosure (classification → audit → root cause → fix), early exit
Early exit: If the failure type is immediately obvious from the description (e.g., "max_tokens too low causing truncation"), skip to Step 6 and propose the fix directly.