Reasoning Verifier

Reasoning Verifier

Verify reasoning using the Reverse Chain-of-Thought (RCoT) methodology to detect and correct errors in LLM-generated solutions by systematically comparing what the problem stated versus what the solution assumed.

When to Use

• After complex multi-step reasoning or analysis tasks
• When solution correctness is critical
• To catch overlooked conditions from the original problem
• To identify hallucinated assumptions not present in the original problem
• Before finalizing recommendations or decisions based on LLM analysis

RCoT Methodology

Follow these steps in order:

Step 1: Problem Decomposition

Extract all conditions from the original problem:

1. Read the original problem statement carefully
2. List every explicit condition, constraint, and requirement
3. Note any implicit conditions that can be logically inferred
4. Number each condition for reference

Format:

## Original Conditions

1. [Condition from original problem]
2. [Condition from original problem]
3. [Condition from original problem]
...

Step 2: Solution Reconstruction

Analyze the provided solution and reconstruct what problem it appears to solve:

1. Read through the solution's reasoning steps
2. Identify every assumption the solution made
3. List conditions the solution used in its reasoning
4. Note the final answer/conclusion

Format:

## Reconstructed Conditions (from solution)

1. [Condition the solution assumed]
2. [Condition the solution assumed]
3. [Condition the solution assumed]
...

## Solution's Answer: [final answer]

Step 3: Condition Comparison

Compare the two condition lists to find discrepancies:

Overlooked Conditions: Present in original but NOT used in solution

• These are facts the solution ignored
• May cause incorrect answers if they affected the logic

Hallucinated Conditions: Present in solution but NOT in original

• These are assumptions the solution invented
• Invalid if they cannot be logically deduced from original conditions

For each candidate condition, ask: "Can this be logically deduced from the other condition list?"

Format:

## Comparison Results

### Overlooked Conditions
- [Condition X] from original was not addressed
  - Impact: [How this affects the solution]

### Hallucinated Conditions
- [Condition Y] was assumed but not stated
  - Validity: [Can this be deduced? YES/NO]
  - Impact: [How this affects the solution]

Step 4: Error Classification

Classify the severity of each discrepancy:

• Critical: Changes the final answer
• Major: Affects intermediate reasoning significantly
• Minor: Technical oversight with no impact on answer
• None: Solution is correct

Step 5: Revision Prompt Generation

If errors are found, generate targeted correction prompts:

For overlooked conditions:

You have ignored some real conditions:
1. [overlooked condition]
Here are detailed reasons:
1. [Why this condition matters and how it affects the answer]

For hallucinated conditions:

You have assumed conditions not in the problem:
1. [hallucinated condition]
This cannot be logically derived because:
1. [Why this assumption is invalid]

Step 6: Revised Solution

If errors were found, provide the corrected solution:

1. Acknowledge the specific mistakes
2. Incorporate all overlooked conditions
3. Remove invalid hallucinated assumptions
4. Show corrected reasoning step-by-step
5. Provide the revised answer

Step 7: Iteration Gate

After producing the revised solution, apply this stopping rule:

If Severity == Critical or Major:
  Re-run from Step 1 using the Revised Solution as the new "provided solution"
  Increment iteration counter (start at 1, cap at 3)
  If iteration counter == 3: halt regardless of severity, flag as "max iterations reached"
Else (Severity == Minor or None):
  Halt — output final verified solution + iteration count
  Label verdict: "No critical gaps remain"

Do NOT target "100% confidence" as a stopping criterion — that framing is unachievable and produces false certainty. The correct stopping condition is no critical or major gaps remain.

Output Format

# RCoT Verification Report

## Problem Summary
[Brief description of the problem being verified]

## Original Conditions
1. [condition]
2. [condition]
...

## Reconstructed Conditions (from solution)
1. [condition]
2. [condition]
...

## Comparison Results

### Overlooked Conditions
[List or "None found"]

### Hallucinated Conditions
[List or "None found"]

## Verdict: [CORRECT / NEEDS REVISION]

## Severity: [Critical / Major / Minor / None]

---

[If NEEDS REVISION:]

## Revision Prompt

[Generated prompt to correct the solution]

## Revised Solution

[Corrected answer with proper reasoning]

Example

Original Problem: "Mary has 40 window ledges. She has 2 potted plants on each ledge. Yesterday, she received 18 new potted plants from her favorite nursery. She decided to give away 1 potted plant from each ledge. How many potted plants will Mary remain with?"

Original Solution: "Mary has 2 x 40 = 80 plants. After giving away 1 from each ledge, she has 80 - 40 = 40 plants."

RCoT Analysis:

• Overlooked: "She received 18 new potted plants" - this condition was completely ignored
• The solution is INCORRECT

Revised Solution: "Mary starts with 2 x 40 = 80 plants. She receives 18 new plants, giving her 80 + 18 = 98 plants. After giving away 1 from each of 40 ledges, she has 98 - 40 = 58 plants."