Correction Detection System

Correction Detection System

The correction detection system automatically captures user corrections and feedback, enabling continuous improvement of agents and skills through a two-stage workflow.

Overview

Correction detection implements a two-stage learning workflow:

Architecture

User Message
      │
      ▼
┌─────────────────────┐
│ Pattern Detection   │ → Regex matching against known patterns
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ Confidence Scoring  │ → Combine pattern weights + context
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ Value Extraction    │ → Extract old/new values from captures
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ Target Inference    │ → Determine where correction applies
└─────────────────────┘
      │
      ▼
┌─────────────────────┐
│ Storage (pending)   │ → Store for /reflect review
└─────────────────────┘

Correction Patterns

The system recognizes various correction pattern types:

Pattern Detection

import { matchCorrectionPatterns } from 'copilot-memory/tools/correction-tools';

const message = 'Actually, use TypeScript instead of JavaScript';
const matches = matchCorrectionPatterns(message);

// Returns:
// [{
//   patternId: 'actually-instead',
//   type: 'replacement',
//   matchedText: 'Actually, use TypeScript instead of JavaScript',
//   position: { start: 0, end: 46 },
//   captures: { group1: 'TypeScript', group2: 'JavaScript' }
// }]

Confidence Scoring

Confidence is calculated from multiple factors:

Score Interpretation

API

import { calculateConfidence } from 'copilot-memory/tools/correction-tools';

const matches = matchCorrectionPatterns(message);
const confidence = calculateConfidence(
  matches,
  message,
  previousAgentOutput  // Optional context
);
// Returns: 0.0 - 1.0

Value Extraction

The system extracts old and new values from matched patterns:

import { extractValues } from 'copilot-memory/tools/correction-tools';

const message = 'Not src/index.ts, but src/main.ts';
const matches = matchCorrectionPatterns(message);
const { oldValue, newValue } = extractValues(matches);

// oldValue: 'src/index.ts'
// newValue: 'src/main.ts'

Extraction Priority

1. Explicit correction patterns (highest priority)
2. Replacement patterns
3. Negation patterns
4. Factual error patterns
5. Clarification patterns
6. Preference patterns

Target Inference

Corrections are routed to appropriate targets based on context:

Agent-Based Routing

Keyword-Based Routing

Full Detection Flow

import { detectCorrections } from 'copilot-memory/tools/correction-tools';

const result = detectCorrections({
  userMessage: 'Correction: use async/await instead of callbacks',
  previousAgentOutput: 'Using callbacks for async...',
  agentId: 'me',
  threshold: 0.5
}, 'project-id');

// Returns:
// {
//   detected: true,
//   corrections: [{
//     id: 'uuid',
//     originalContent: 'Using callbacks for async...',
//     correctedContent: 'async/await',
//     rawUserMessage: 'Correction: use async/await...',
//     matchedPatterns: [...],
//     target: 'skill',
//     targetId: 'agent-me',
//     confidence: 0.95,
//     status: 'pending',
//     expiresAt: '...'  // 7 days
//   }],
//   patternMatchCount: 1,
//   maxConfidence: 0.95,
//   suggestedAction: 'auto_capture'
// }

/reflect Command

The /reflect command provides a review interface for pending corrections.

Usage

# Review all pending corrections
/reflect

# Filter by status
/reflect --status pending
/reflect --status approved

# Filter by agent
/reflect --agent me

# Include applied corrections
/reflect --include-applied

Review Actions

Deduplication

Use --dedupe to consolidate similar corrections:

/reflect --dedupe

This groups corrections with similar content and allows bulk approval/rejection.

Correction Routing

After approval, corrections are routed to their target:

To Skill/Agent Files

import { routeCorrection } from 'copilot-memory/tools/correction-tools';

const route = routeCorrection(db, correctionId);
// Returns:
// {
//   correctionId: '...',
//   target: 'skill',
//   targetPath: '.claude/agents/me.md',
//   responsibleAgent: 'me',
//   confidence: 0.95,
//   applyInstructions: 'Update .claude/agents/me.md:\n- Find section...'
// }

To Memory

For memory and preference targets, corrections are stored via Memory Copilot:

// memory_store is called with:
{
  type: 'lesson',  // or 'context' for preferences
  content: 'User prefers: async/await over callbacks',
  tags: ['correction', 'user-feedback']
}

MCP Tools

correction_detect

const result = await correction_detect({
  userMessage: 'Actually, use the other approach',
  previousAgentOutput: '...',
  agentId: 'me',
  threshold: 0.5,
  autoStore: false  // If true, stores automatically above threshold
});

correction_list

const corrections = await correction_list({
  status: 'pending',
  agentId: 'me',
  target: 'skill',
  limit: 20,
  includeExpired: false
});

correction_update

await correction_update({
  correctionId: '...',
  status: 'approved',  // or 'rejected'
  rejectionReason: 'False positive'  // If rejecting
});

Database Schema

Corrections are stored in the Memory Copilot database:

Best Practices

1. Threshold Tuning: Start with 0.5 threshold, adjust based on false positive rate
2. Review Regularly: Use /reflect periodically to prevent expiration
3. Agent Context: Provide agentId for better target inference
4. Previous Output: Include previousAgentOutput for context-aware detection

Troubleshooting

Missing Detections

1. Check if message matches any pattern: matchCorrectionPatterns(message)
2. Verify threshold isn't too high
3. Try lowering threshold parameter

False Positives

1. Increase threshold (0.6-0.7)
2. Review and reject via /reflect
3. Check if patterns are too broad

Routing Issues

1. Provide agentId for agent-based routing
2. Check keyword detection for target inference
3. Use forceTarget parameter if needed

Related Documentation

• Lifecycle Hooks
• Skill Evaluation
• Memory Copilot