Code Duplication

Code Duplication Analysis Skill

Comprehensive code duplication detection and analysis for all programming languages.

Use this skill to identify technical debt from code duplication, quantify duplication metrics, and receive actionable refactoring suggestions.

Usage

# Analyze entire codebase
/code-duplication

# Analyze specific directory
/code-duplication src/components

# Analyze with custom configuration
/code-duplication --config .duplication-config.json

# Generate JSON output
/code-duplication --format json --output duplication-report.json

Converging sweep (loop-until-dry)

cli.py is one pass: it runs each detector once and reports what that run found. When you need confidence that nothing is left, use the converging workflow instead — workflows/code-duplication-loop-until-dry.js. It wraps these same analyzers (it does not reimplement them) in rounds of finder agents, and keeps going until K consecutive rounds surface nothing new (default K=2), with a hard round cap as a backstop.

Workflow({
  scriptPath: '~/.claude/workflows/code-duplication-loop-until-dry.js',
  args: {
    repo_root: '/path/to/repo',
    targets: ['src'],              // default ['.']
    k_empty: 2,                    // consecutive empty rounds required to stop
    max_rounds: 5,                 // hard backstop; exceeding it is reported, never hidden
    max_judged_per_round: 4,
    run_started_at: '<ISO timestamp — the sandbox has no clock>',
  },
})

Always by scriptPath, never by name: user-level workflows do not resolve in the named registry (docs/graph-orchestration.md §6).

Each round is deduped against every finding ever seen — including ones an adversarial judge already rejected — not just against the survivors. That is the difference between a loop that runs dry and one that rediscovers, re-judges and re-rejects the same dead ends every round forever (FRS FR-16). The returned seen_ledger is the audit trail: every fingerprint, its disposition, the round it was first seen in, and every later round that re-surfaced it and had it dropped.

Novel findings go through a refute-by-default judge before they are reported, so result.findings holds only what survived; rejected and never-judged findings are returned separately under result.rejected and result.unverified rather than folded in. Anything a cap dropped — the round cap, the per-batch cap, the judging budget, a dead finder node — is named in result.disclosures.

Which one to run:

Do not reach for the workflow to re-run the metrics: one pass already gives you those, and the workflow's value is convergence and verification, not numbers.

What This Skill Does

Performs comprehensive duplicate detection to answer:

• What code blocks are duplicated across the codebase?
• What is the overall duplication percentage?
• Which files have the most duplication?
• How much LOC could be saved through refactoring?
• What specific refactoring techniques should be applied?

Detection Types

1. Exact Duplicates

Identical code blocks (copy-pasted code)

• Threshold: 5-10 lines minimum
• Ignores whitespace and comments
• Groups all instances together

2. Structural Duplicates

Similar logic with minor variations

• Uses AST-based comparison for Python/JavaScript
• Detects code with different variable names but same structure
• Similarity scoring with configurable threshold

3. Pattern Duplicates

Repeated coding patterns that could be abstracted

• Common anti-patterns (repeated try-catch, validation logic, etc.)
• Opportunities for extract function/class refactoring
• Pattern catalog extensible

Report Output

Markdown Report (Default)

# Code Duplication Analysis Report

## Executive Summary
- Total LOC analyzed: 12,450
- Duplicate LOC: 1,834 (14.7%)
- Duplicate blocks found: 47
- Estimated LOC reduction: 1,200-1,500

## Top Offenders
1. src/services/user_service.py - 23% duplication (340 LOC)
2. src/api/handlers.py - 18% duplication (245 LOC)
...

## Detailed Findings
### Duplicate Block #1 (Exact)
**Instances**: 4 files
**LOC per instance**: 23 lines
**Potential savings**: 69 lines

**Locations**:
- src/services/user_service.py:145-167
- src/services/order_service.py:89-111
...

**Refactoring Suggestion**:
Extract common authentication logic into shared function:
...

JSON Report (--format json)

{
  "summary": {
    "total_files": 156,
    "total_loc": 12450,
    "duplicate_loc": 1834,
    "duplication_percentage": 14.7,
    "duplicate_blocks": 47
  },
  "duplicates": [...],
  "top_offenders": [...],
  "heatmap": {...}
}

Configuration

Create .duplication-config.json in your project root:

{
  "min_lines": 5,
  "exclude_patterns": ["**/test_*.py", "**/migrations/*.py"],
  "similarity_threshold": 0.85,
  "ignore_comments": true,
  "ignore_whitespace": true,
  "languages": ["python", "javascript", "typescript"]
}

Quality Metrics

This skill provides:

• Duplication Percentage: % of codebase that is duplicated
• Top Offenders: Files ranked by duplication amount
• File Heatmap: Visual representation of duplication distribution
• Cleanup Potential: Estimated LOC reduction from refactoring

Technical Details

• Zero external dependencies: Pure Python 3.8+ stdlib
• Multi-language support: Python, JavaScript, TypeScript, and more
• Performance: < 30 seconds for 10,000 LOC
• Memory efficient: Streaming file processing
• Git integration: Supports incremental mode (analyze only changed files)

Implementation Status

✅ Complete — All features implemented and tested

• ✅ Data models and configuration loader
• ✅ Exact duplicate detection (hash-based)
• ✅ Structural duplicate detection (AST-based)
• ✅ Pattern duplicate detection (12 anti-patterns)
• ✅ Metrics calculation and trend analysis
• ✅ Report generation (Markdown + CSV)
• ✅ CLI interface with full argument parsing
• ✅ Git integration (incremental analysis)
• ✅ Heatmap visualization
• ✅ Refactoring suggestion engine