Cfn Loop Output Processing

CFN Loop Output Processing - Unified TypeScript Module

Version: 1.0.0 Status: Production Purpose: Type-safe, consolidated output processing for Loop 2 (Validators) and Loop 3 (Implementers) agents

Overview

Problem Solved

Previously, output processing was scattered across multiple bash scripts with duplicate logic:

• Loop 2 (validators): cfn-loop2-output-processing/parse-feedback.sh
• Loop 3 (implementers): cfn-loop3-output-processing/parse-confidence.sh
• Duplicate confidence extraction logic
• No type safety
• Difficult to maintain and extend

Solution

Single TypeScript module (cfn-loop-output-processing) providing:

• Type-safe interfaces for all result types
• Unified confidence extraction with 5+ parsing patterns
• Robust feedback parsing with severity categorization
• Consensus calculation from multiple validators
• CLI entry points for orchestrator integration
• 90%+ test coverage with comprehensive test suite

Architecture

Module Structure

.claude/skills/cfn-loop-orchestration-v2/lib/output/
├── src/
│   ├── output-processor.ts          # Core module (type-safe)
│   └── cli/
│       ├── process-loop3.ts         # Loop 3 CLI tool
│       └── process-loop2.ts         # Loop 2 CLI tool
├── tests/
│   └── output-processor.test.ts     # Comprehensive test suite (90%+ coverage)
├── SKILL.md                         # This file
└── package.json                     # Dependencies

Key Types

// Loop 3 Result (Implementers)
interface Loop3Result {
  agentId: string;
  confidence: number;
  confidenceSource: 'explicit' | 'calculated' | 'fallback';
  filesChanged: number;
  deliverables: string[];
  testsPassedCount?: number;
  testsFailed?: number;
  passRate?: number;
  output: string;
  iteration: number;
  timestamp: string;
}

// Loop 2 Result (Validators)
interface Loop2Result {
  validatorId: string;
  score: number;
  scoreSource: 'explicit' | 'calculated' | 'qualitative';
  issues: FeedbackItem[];
  criticalCount: number;
  warningCount: number;
  suggestionCount: number;
  recommendations: string[];
  output: string;
  iteration: number;
  timestamp: string;
}

// Consensus Result
interface ConsensusResult {
  averageScore: number;
  threshold: number;
  passed: boolean;
  validatorCount: number;
  scoredCount: number;
  minScore: number;
  maxScore: number;
  summary: string;
  details: {
    criticalIssuesTotal: number;
    warningIssuesTotal: number;
    suggestionsTotal: number;
  };
}

Core Functions

Confidence Extraction

parseConfidence(output, config?)

Extracts confidence score using 5-pattern fallback approach.

Patterns (priority order):

1. Explicit header: ## Validation Confidence: 0.85
2. Generic field: confidence: 0.92 or Confidence: 0.92
3. Score field: Score: 0.78
4. Percentage: 92% or 92 percent
5. Parentheses: (0.87)
6. Qualitative: high confidence → 0.90, medium → 0.75, low → 0.50

Returns:

{
  score: number;          // 0.0-1.0
  source: 'explicit' | 'qualitative' | 'none';
}

Examples:

parseConfidence('Confidence: 0.85');           // { score: 0.85, source: 'explicit' }
parseConfidence('92%');                        // { score: 0.92, source: 'explicit' }
parseConfidence('high confidence');            // { score: 0.90, source: 'qualitative' }
parseConfidence('No score provided');          // { score: 0.0, source: 'none' }

Feedback Extraction

extractFeedback(output)

Extracts feedback items from validator output, categorized by severity.

Supported formats:

• Markdown sections: ### CRITICAL Issues\n- Issue text
• Inline format: CRITICAL: Issue text
• Multiple bullet styles: -, *, •

Returns:

FeedbackItem[] = [
  { severity: 'CRITICAL', text: 'Missing error handling' },
  { severity: 'WARNING', text: 'Performance issue' },
  { severity: 'SUGGESTION', text: 'Add documentation' }
]

Example:

### CRITICAL Issues
- SQL injection vulnerability
- Missing authentication

### WARNING Issues
- Performance concern in loop

### SUGGESTION Items
- Add unit tests

extractRecommendations(output)

Extracts recommendations/suggestions from validator output.

Supported headers: Recommendations:, Recommendation:, Suggestions:

Returns: string[] of recommendations

Consensus Calculation

calculateConsensus(results, threshold?)

Calculates consensus from multiple validator results.

Parameters:

• results: Array of Loop2Result
• threshold: Minimum average score to pass (default: 0.70)

Returns:

{
  averageScore: number;
  threshold: number;
  passed: boolean;
  validatorCount: number;
  scoredCount: number;
  minScore: number;
  maxScore: number;
  summary: string;
  details: {
    criticalIssuesTotal: number;
    warningIssuesTotal: number;
    suggestionsTotal: number;
  };
}

Example:

const consensus = calculateConsensus([
  { validatorId: 'v1', score: 0.9, ... },
  { validatorId: 'v2', score: 0.85, ... }
], 0.80);

// Result: { averageScore: 0.875, passed: true, ... }

Loop-Specific Parsers

parseLoop3Output(agentOutput, agentId, iteration?, gitStatus?)

Complete Loop 3 parser combining all extraction logic.

Features:

• Extracts explicit confidence or calculates fallback
• Counts files changed from git status
• Extracts test results if present
• Validates all data

Example:

const result = parseLoop3Output(
  agentOutput,
  'coder-1',
  1,
  { before: '...', after: '...' }
);

// Result: Loop3Result with confidence, deliverables, test info

parseLoop2Output(validatorOutput, validatorId, iteration?)

Complete Loop 2 parser for validator feedback.

Features:

• Extracts confidence score
• Parses feedback by severity
• Extracts recommendations
• Counts issues by category

Example:

const result = parseLoop2Output(
  validatorOutput,
  'reviewer-1',
  1
);

// Result: Loop2Result with score, issues, recommendations

CLI Tools

Loop 3 Processor

npx ts-node src/cli/process-loop3.ts [OPTIONS]

Options:

• --agent-id <id> - Agent identifier (required)
• --output <text> - Agent output text (required)
• --output-file <path> - Alternative: read from file
• --iteration <n> - Iteration number (default: 1)
• --files-changed <n> - Number of files changed
• --deliverables <file,file> - Comma-separated files

Output: JSON with Loop3Result

Example:

npx ts-node src/cli/process-loop3.ts \
  --agent-id "coder-1" \
  --output "Implementation complete. Confidence: 0.85" \
  --files-changed 5

# Output:
# {
#   "agentId": "coder-1",
#   "confidence": 0.85,
#   "confidenceSource": "explicit",
#   "filesChanged": 5,
#   ...
# }

Loop 2 Processor (Single Validator)

npx ts-node src/cli/process-loop2.ts [OPTIONS]

Options:

• --validator-id <id> - Validator identifier (required)
• --output <text> - Validator output text (required)
• --output-file <path> - Alternative: read from file
• --iteration <n> - Iteration number (default: 1)

Output: JSON with Loop2Result

Example:

npx ts-node src/cli/process-loop2.ts \
  --validator-id "reviewer-1" \
  --output "Validation: confidence 0.92"

# Output:
# {
#   "validatorId": "reviewer-1",
#   "score": 0.92,
#   ...
# }

Loop 2 Consensus Mode

npx ts-node src/cli/process-loop2.ts --consensus [OPTIONS]

Options:

• --consensus - Enable consensus mode
• --results-file <path> - JSON file with Loop2Result array (required)
• --threshold <n> - Minimum score to pass (default: 0.70)

Output: JSON with ConsensusResult

Example:

npx ts-node src/cli/process-loop2.ts \
  --consensus \
  --results-file ./validator-results.json \
  --threshold 0.75

# Output:
# {
#   "averageScore": 0.88,
#   "passed": true,
#   "summary": "PASS: 88% consensus from 3 validators...",
#   ...
# }

Integration with Orchestrator

Loop 3 Integration

# From orchestrator: process implementer output
RESULT=$(npx ts-node $HOME/.claude/skills/cfn-loop-orchestration-v2/lib/output/src/cli/process-loop3.ts \
  --agent-id "$AGENT_ID" \
  --output "$AGENT_OUTPUT" \
  --iteration "$ITERATION")

CONFIDENCE=$(echo "$RESULT" | jq -r '.confidence')
FILES_CHANGED=$(echo "$RESULT" | jq -r '.filesChanged')

# Report to Redis
redis-cli LPUSH "swarm:${TASK_ID}:${AGENT_ID}:confidence" "$CONFIDENCE"
redis-cli LPUSH "swarm:${TASK_ID}:${AGENT_ID}:done" "complete"

Loop 2 Integration (Single Validator)

# Process validator output
RESULT=$(npx ts-node $HOME/.claude/skills/cfn-loop-orchestration-v2/lib/output/src/cli/process-loop2.ts \
  --validator-id "$VALIDATOR_ID" \
  --output "$VALIDATOR_OUTPUT" \
  --iteration "$ITERATION")

SCORE=$(echo "$RESULT" | jq -r '.score')

# Store for later consensus calculation
echo "$RESULT" >> ./validator-results.json

Loop 2 Integration (Consensus)

# Calculate consensus from all validators
CONSENSUS=$(npx ts-node $HOME/.claude/skills/cfn-loop-orchestration-v2/lib/output/src/cli/process-loop2.ts \
  --consensus \
  --results-file ./validator-results.json \
  --threshold "$CONSENSUS_THRESHOLD")

PASSED=$(echo "$CONSENSUS" | jq -r '.passed')

if [ "$PASSED" = "true" ]; then
  echo "Validators consensus PASSED"
else
  echo "Validators consensus FAILED - average score below threshold"
fi

Confidence Extraction Patterns

NOTE: this 5-pattern extraction ladder is RECOVERY-ONLY. It exists to salvage confidence from malformed agent output. Orchestrator prompts must demand the JSON contract as primary (agents report an explicit JSON block with a decimal confidence field, computed by the mechanical rubric in lib/validation/SKILL.md). If you find yourself relying on patterns 3-5, fix the spawn prompt, not the parser.

Priority Order

1. Explicit Header (Priority 1)

   • ## Validation Confidence: 0.85
   • Highest precision

2. Generic Field (Priority 2)

   • confidence: 0.92
   • Confidence: 0.92
   • Score: 0.78

3. Percentage (Priority 3)

   • 92%
   • 92 percent

4. Parentheses (Priority 4)

   • (0.87)
   • Context-dependent

5. Qualitative (Priority 5)

   • high confidence → 0.90
   • medium confidence → 0.75
   • low confidence → 0.50

Fallback Strategy

If no explicit confidence found:

1. Check qualitative mappings
2. For Loop 3 only: Calculate from file changes + tests
3. Return 0.0 with source="none"

Feedback Categorization

CRITICAL Issues

• Blocking problems
• Security vulnerabilities
• Fundamental functionality breaks
• Must be fixed before proceeding

WARNING Issues

• Important non-blocking problems
• Performance concerns
• Maintainability issues
• Should be addressed

SUGGESTION Items

• Optional improvements
• Code style recommendations
• Minor optimizations
• Nice-to-have enhancements

Test Coverage

Test Suite

File: tests/output-processor.test.ts Coverage: 90%+ (lines, functions, branches)

Test Categories

• Confidence extraction (8 tests): All 5 patterns, qualitative, edge cases
• Feedback extraction (5 tests): Markdown sections, inline format, edge cases
• Recommendations (4 tests): Multiple header formats, empty cases
• Fallback calculation (6 tests): File change thresholds, test impacts
• Validation (4 tests): Range validation, NaN handling
• Loop 3 parsing (4 tests): Complete workflow, git integration
• Loop 2 parsing (3 tests): Complete workflow, categorization
• Consensus (6 tests): Multiple validators, edge cases, aggregation
• Default detection (3 tests): Default output identification
• Serialization (3 tests): JSON round-trip
• Integration (2 tests): Complete workflows

Running Tests

# Install dependencies
npm install

# Run all tests with coverage
npm run test

# Run specific test suite
npm test -- --testNamePattern="parseConfidence"

# Watch mode for development
npm test -- --watch

Migration Guide

From Bash to TypeScript

Old (Bash):

CONFIDENCE=$(./parse-confidence.sh "$AGENT_OUTPUT")
DELIVERABLES=$(./verify-deliverables.sh --before "$BEFORE" --after "$AFTER")

New (TypeScript):

import { parseConfidence, parseLoop3Output } from './output-processor';

const { score, source } = parseConfidence(agentOutput);
const result = parseLoop3Output(agentOutput, agentId, iteration, gitStatus);

CLI (Same interface):

npx ts-node src/cli/process-loop3.ts \
  --agent-id "coder-1" \
  --output "$AGENT_OUTPUT"

Deprecation Timeline

Immediate (Now):

• cfn-loop2-output-processing/parse-feedback.sh → Use TypeScript module
• cfn-loop3-output-processing/parse-confidence.sh → Use TypeScript module
• cfn-loop3-output-processing/calculate-confidence.sh → Use TypeScript module

30 days: Bash scripts continue to work (deprecated status) 60 days: Orchestrator fully migrated to TypeScript 90 days: Bash scripts removed

Benefits

Error Handling

Invalid Input

// Empty input
parseConfidence('');                    // { score: 0.0, source: 'none' }

// Invalid JSON in results file
const results = parseJson(invalidJson); // null

// Out of range score
isValidConfidence(1.5);                 // false

CLI Error Messages

# Missing required argument
$ npx ts-node src/cli/process-loop3.ts
Error: --agent-id is required

# File not found
$ npx ts-node src/cli/process-loop3.ts --output-file missing.txt
Error: Output file not found: missing.txt

Performance

• Confidence extraction: <1ms
• Feedback parsing: <2ms
• Consensus calculation: <5ms (10 validators)
• Total CLI execution: <100ms

Memory efficient with minimal allocations.

Backward Compatibility

Breaking Changes

None. This module consolidates without changing external interfaces.

CLI Compatibility

All CLI tools accept same arguments as predecessor bash scripts.

Integration Points

Orchestrator integration remains identical - only internal implementation changes.

Related Documentation

• Agent Output Processing (.claude/skills/cfn-agent-lifecycle/lib/output/README.md) - Universal pattern
• Loop Orchestration (.claude/skills/cfn-loop-orchestration-v2/lib/orchestrator/SKILL.md) - Coordinator integration
• CFN Loop Architecture (docs/CFN_LOOP_ARCHITECTURE.md) - System design

Version History

1.0.0 (2025-11-19)

• Initial TypeScript consolidation
• Unified confidence extraction (5+ patterns)
• Robust feedback parsing
• Consensus calculation
• Type-safe interfaces
• 90%+ test coverage
• CLI entry points for orchestrator
• Deprecates 3 bash scripts

Summary

This skill provides type-safe, consolidated output processing for all CFN Loop agents. By consolidating previously scattered bash scripts into a single TypeScript module, we achieve:

• Type Safety: Compile-time error detection
• Maintainability: Single source of truth for parsing logic
• Testability: 90%+ test coverage with comprehensive suite
• Performance: Optimized parsing with multi-pattern fallbacks
• Documentation: Complete API documentation with examples
• Backward Compatibility: Identical CLI interfaces

The module is production-ready and replaces cfn-loop2-output-processing and cfn-loop3-output-processing bash skills.