Reasoning Path Confidence

Technique: Reasoning Path Confidence — Efficient Test-Time Scaling

LLMs achieve strong performance on complex reasoning tasks through test-time sampling strategies like self-consistency, but this approach requires extensive sampling from the full probability distribution. However, sampling introduces high estimation error while perplexity-based filtering suffers from high modeling error. Most reasoning paths are ineffective noise.

RPC bridges these approaches by recognizing that self-consistency and perplexity capture complementary signals. Rather than choosing between them, the technique combines perplexity consistency to detect unreliable reasoning branches with reasoning pruning to eliminate low-probability paths before ensemble voting.

Core Concept

RPC operates on two key insights:

• Perplexity Consistency: Path perplexity correlates with reasoning quality—high perplexity indicates uncertain or incoherent reasoning steps
• Reasoning Pruning: Eliminating low-probability reasoning paths reduces noise without degrading the signal from high-quality paths

The combination achieves exponential convergence improvement (from linear to exponential) compared to self-consistency alone while reducing sampling costs by 50%.

Architecture Overview

• Probability Distribution Analysis: Track per-step token probabilities during chain-of-thought generation
• Perplexity Scoring: Compute Shannon entropy over next-token distributions to detect uncertainty spikes
• Path Filtering: Prune reasoning paths where cumulative perplexity exceeds a learned threshold
• Ensemble Voting: Apply standard self-consistency voting only on surviving high-quality paths
• Convergence Acceleration: Fewer but higher-quality samples accelerate accuracy improvement

Implementation Steps

The core algorithm computes perplexity per step and filters paths before ensemble aggregation.

import numpy as np
from collections import defaultdict

def reasoning_path_confidence(
    reasoning_paths,
    step_log_probs,
    perplexity_threshold=2.5,
    top_k_paths=None
):
    """
    RPC filtering: prune low-confidence reasoning paths before ensemble voting.

    Args:
        reasoning_paths: list of reasoning strings (complete chains-of-thought)
        step_log_probs: list of lists, log probabilities per step in each path
        perplexity_threshold: entropy cutoff for path acceptance
        top_k_paths: if set, keep only top-k paths by score

    Returns:
        filtered_paths: paths surviving confidence filtering
        path_scores: confidence scores for each surviving path
    """
    # Compute perplexity (entropy) per step for each path
    perplexity_scores = []
    for i, log_probs in enumerate(step_log_probs):
        # Average entropy across steps in this reasoning path
        step_entropies = [-lp for lp in log_probs]  # Higher negLP = higher entropy
        path_perplexity = np.mean(step_entropies)
        perplexity_scores.append(path_perplexity)

    # Filter paths by perplexity threshold
    surviving_indices = [
        i for i, score in enumerate(perplexity_scores)
        if score <= perplexity_threshold
    ]

    filtered_paths = [reasoning_paths[i] for i in surviving_indices]
    filtered_scores = [perplexity_scores[i] for i in surviving_indices]

    # Optional: keep only top-k highest-confidence paths
    if top_k_paths and len(filtered_paths) > top_k_paths:
        top_indices = np.argsort(filtered_scores)[:top_k_paths]
        filtered_paths = [filtered_paths[i] for i in top_indices]
        filtered_scores = [filtered_scores[i] for i in top_indices]

    return filtered_paths, filtered_scores


def ensemble_reasoning_with_rpc(
    model,
    prompt,
    num_samples=20,
    perplexity_threshold=2.5
):
    """
    Generate multiple reasoning paths, apply RPC filtering, then ensemble.
    """
    paths = []
    log_probs_list = []

    for _ in range(num_samples):
        # Generate path and collect per-step log probabilities
        path, step_lps = model.generate_with_logprobs(
            prompt, max_steps=50
        )
        paths.append(path)
        log_probs_list.append(step_lps)

    # Apply RPC filtering
    filtered_paths, scores = reasoning_path_confidence(
        paths, log_probs_list, perplexity_threshold
    )

    # Extract answers and vote
    answers = extract_answers(filtered_paths)
    final_answer = majority_vote(answers)

    return final_answer, len(filtered_paths), num_samples

Perplexity threshold tuning: Start at 2.5 and adjust based on validation set. Higher threshold = more paths survive = more computation, lower threshold = stronger filtering = fewer samples needed.

Practical Guidance

When to Use:

• You have a large sampling budget and want to reduce inference cost without losing accuracy
• Reasoning tasks have clear right/wrong answers for ensemble voting
• You can log per-step token probabilities from your model

When NOT to Use:

• Open-ended generation without verifiable correct answers
• Tasks where perplexity doesn't correlate with answer quality
• Online inference with strict latency constraints (pruning adds overhead)

Common Pitfalls:

• Setting perplexity threshold too low → filters out valid paths, degrades accuracy
• Not normalizing entropy across variable-length reasoning chains
• Applying threshold uniformly to all task difficulties (easier tasks may have lower perplexity)

Reference

NeurIPS 2025 | A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning