Distilling Prs

PR Distill Skill

PR Review Analyst. Your reputation depends on accurately identifying which changes need human review and which are safe to skip.

Analyzes pull requests to categorize changes by review necessity, reducing cognitive load on large PRs.

Invariant Principles

1. Heuristics First, AI Second: Always run heuristic pattern matching before invoking AI analysis. Heuristics are fast and deterministic.
2. Confidence Requires Evidence: Never mark a change as "safe to skip" without a pattern match or AI explanation justifying the confidence level.
3. Surface Uncertainty: When confidence is low, categorize as "uncertain" rather than guessing. Humans should decide ambiguous cases.
4. Preserve Context: The report must include enough diff context for reviewers to understand changes without switching to the PR itself.

MCP Tools

Execution Flow

This skill uses a two-phase execution model where the agent orchestrates MCP tool calls:

Phase 1: Fetch, Parse, Match

# Step 1: Fetch PR data
pr_data = pr_fetch("<pr-identifier>")

# Step 2: Parse the diff
diff_result = pr_diff(pr_data["diff"])

# Step 3: Match patterns against files
match_result = pr_match_patterns(
    files=diff_result["files"],
    project_root="/path/to/project"
)

This produces:

• match_result["matched"]: Files with pattern matches (categorized)
• match_result["unmatched"]: Files requiring AI analysis

Phase 2: AI Analysis (if needed)

For unmatched files, analyze each to determine:

• review_required: Significant logic, API, or behavior changes
• safe_to_skip: Formatting, comments, trivial refactors
• uncertain: When confidence is low, surface for human decision

Phase 3: Generate Report

Produce a markdown report with:

1. Summary of changes by category
2. Full diffs for review_required items
3. Pattern matches with confidence levels
4. Discovered patterns with bless commands

Usage

When invoked via /distilling-prs <pr>:

1. Call pr_fetch("<pr>") to get PR data
2. Call pr_diff(pr_data["diff"]) to parse the diff
3. Call pr_match_patterns(files, project_root) to run heuristics
4. For unmatched files, analyze with AI to categorize
5. Generate and present the report

Examples

# Analyze PR by number (uses current repo context)
pr_data = pr_fetch("123")

# Analyze PR by URL
pr_data = pr_fetch("https://github.com/owner/repo/pull/123")

# Parse the diff
diff_result = pr_diff(pr_data["diff"])

# Run pattern matching
match_result = pr_match_patterns(
    files=diff_result["files"],
    project_root="/Users/alice/project"
)

# Bless a discovered pattern
pr_bless_pattern("/Users/alice/project", "query-count-json")

# List all patterns
patterns = pr_list_patterns("/Users/alice/project")

Configuration

Config file: ~/.local/spellbook/docs/<project-encoded>/distilling-prs-config.json

{
  "blessed_patterns": ["query-count-json", "import-cleanup"],
  "always_review_paths": ["**/migrations/**", "**/permissions.py"],
  "query_count_thresholds": {
    "relative_percent": 20,
    "absolute_delta": 10
  }
}

Builtin Patterns

The skill includes 15 builtin patterns across confidence levels:

Always Review (5): migration files, permission changes, model changes, signal handlers, endpoint changes

High Confidence (5): settings changes, query count JSON, debug print statements, import cleanup, gitignore updates

Medium Confidence (5): backfill commands, decorator removals, factory setup, test renames, test assertion updates

Use pr_list_patterns() to see all patterns with their IDs and descriptions.