Check Answer Consistency

Use this to get a cheap, reference-free signal that an LLM answer might be made up, by sampling the same prompt a few times and measuring agreement. Trigger on "is this answer reliable", "flag low-confidence answers", "cheap hallucination check", "confidence score without a ground truth", "self-consistency check". Ships a runnable, tested scorer you can put inline or on sampled traffic.

ContextJet-ai Updated 49 repo stars

File contents

ContextJet-ai/awesome-llm-observability commit 186de296f0

Frequently asked questions

npx get-skill add contextjet-ai/check-answer-consistency