# Code Duplication

> Deep analysis of codebase for code duplication. Detects exact, structural, and pattern-level duplicates, generates comprehensive reports with refactoring suggestions and metrics.

- Skill: `artsmc-claude-dev-agents/code-duplication` (Agent Skill, multi-file: 19 files)
- Install (CLI): `npx skillmds@latest add artsmc-claude-dev-agents/code-duplication`
- Raw SKILL.md: https://api.skillmd.com/api/skills/artsmc-claude-dev-agents/code-duplication/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: artsmc (https://skillmd.com/u/artsmc-claude-dev-agents)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/artsmc-claude-dev-agents/code-duplication

---


# Code Duplication Analysis Skill

Comprehensive code duplication detection and analysis for all programming languages.

**Use this skill to** identify technical debt from code duplication, quantify duplication metrics, and receive actionable refactoring suggestions.

## Usage

```bash
# Analyze entire codebase
/code-duplication

# Analyze specific directory
/code-duplication src/components

# Analyze with custom configuration
/code-duplication --config .duplication-config.json

# Generate JSON output
/code-duplication --format json --output duplication-report.json
```

## Converging sweep (loop-until-dry)

`cli.py` is one pass: it runs each detector once and reports what that run found. When you need
confidence that nothing is *left*, use the converging workflow instead —
`workflows/code-duplication-loop-until-dry.js`. It wraps these same analyzers (it does not
reimplement them) in rounds of finder agents, and keeps going until **K consecutive rounds surface
nothing new** (default K=2), with a hard round cap as a backstop.

```js
Workflow({
  scriptPath: '~/.claude/workflows/code-duplication-loop-until-dry.js',
  args: {
    repo_root: '/path/to/repo',
    targets: ['src'],              // default ['.']
    k_empty: 2,                    // consecutive empty rounds required to stop
    max_rounds: 5,                 // hard backstop; exceeding it is reported, never hidden
    max_judged_per_round: 4,
    run_started_at: '<ISO timestamp — the sandbox has no clock>',
  },
})
```

Always by `scriptPath`, never by `name`: user-level workflows do not resolve in the named registry
(`docs/graph-orchestration.md` §6).

Each round is deduped **against every finding ever seen** — including ones an adversarial judge
already rejected — not just against the survivors. That is the difference between a loop that runs
dry and one that rediscovers, re-judges and re-rejects the same dead ends every round forever
(FRS FR-16). The returned `seen_ledger` is the audit trail: every fingerprint, its disposition, the
round it was first seen in, and every later round that re-surfaced it and had it dropped.

Novel findings go through a refute-by-default judge before they are reported, so `result.findings`
holds only what survived; rejected and never-judged findings are returned separately under
`result.rejected` and `result.unverified` rather than folded in. Anything a cap dropped — the round
cap, the per-batch cap, the judging budget, a dead finder node — is named in `result.disclosures`.

**Which one to run:**

| Use | When |
|---|---|
| `python3 scripts/cli.py <path>` | You want metrics, a heatmap, or a report *now*: duplication %, top offenders, cleanup potential. Deterministic, no agents, seconds. This is the default. |
| `code-duplication-loop-until-dry.js` | You need exhaustiveness you can defend — pre-refactor scoping, a debt audit, "did we get all of it?" — and are willing to spend agents on rounds plus per-finding verification. |

Do not reach for the workflow to re-run the metrics: one pass already gives you those, and the
workflow's value is convergence and verification, not numbers.

## What This Skill Does

Performs comprehensive duplicate detection to answer:
- What code blocks are duplicated across the codebase?
- What is the overall duplication percentage?
- Which files have the most duplication?
- How much LOC could be saved through refactoring?
- What specific refactoring techniques should be applied?

## Detection Types

### 1. Exact Duplicates
Identical code blocks (copy-pasted code)
- Threshold: 5-10 lines minimum
- Ignores whitespace and comments
- Groups all instances together

### 2. Structural Duplicates
Similar logic with minor variations
- Uses AST-based comparison for Python/JavaScript
- Detects code with different variable names but same structure
- Similarity scoring with configurable threshold

### 3. Pattern Duplicates
Repeated coding patterns that could be abstracted
- Common anti-patterns (repeated try-catch, validation logic, etc.)
- Opportunities for extract function/class refactoring
- Pattern catalog extensible

## Report Output

### Markdown Report (Default)

```markdown
# Code Duplication Analysis Report

## Executive Summary
- Total LOC analyzed: 12,450
- Duplicate LOC: 1,834 (14.7%)
- Duplicate blocks found: 47
- Estimated LOC reduction: 1,200-1,500

## Top Offenders
1. src/services/user_service.py - 23% duplication (340 LOC)
2. src/api/handlers.py - 18% duplication (245 LOC)
...

## Detailed Findings
### Duplicate Block #1 (Exact)
**Instances**: 4 files
**LOC per instance**: 23 lines
**Potential savings**: 69 lines

**Locations**:
- src/services/user_service.py:145-167
- src/services/order_service.py:89-111
...

**Refactoring Suggestion**:
Extract common authentication logic into shared function:
...
```

### JSON Report (--format json)

```json
{
  "summary": {
    "total_files": 156,
    "total_loc": 12450,
    "duplicate_loc": 1834,
    "duplication_percentage": 14.7,
    "duplicate_blocks": 47
  },
  "duplicates": [...],
  "top_offenders": [...],
  "heatmap": {...}
}
```

## Configuration

Create `.duplication-config.json` in your project root:

```json
{
  "min_lines": 5,
  "exclude_patterns": ["**/test_*.py", "**/migrations/*.py"],
  "similarity_threshold": 0.85,
  "ignore_comments": true,
  "ignore_whitespace": true,
  "languages": ["python", "javascript", "typescript"]
}
```

## Quality Metrics

This skill provides:
- **Duplication Percentage**: % of codebase that is duplicated
- **Top Offenders**: Files ranked by duplication amount
- **File Heatmap**: Visual representation of duplication distribution
- **Cleanup Potential**: Estimated LOC reduction from refactoring

## Technical Details

- **Zero external dependencies**: Pure Python 3.8+ stdlib
- **Multi-language support**: Python, JavaScript, TypeScript, and more
- **Performance**: < 30 seconds for 10,000 LOC
- **Memory efficient**: Streaming file processing
- **Git integration**: Supports incremental mode (analyze only changed files)

## Implementation Status

✅ **Complete** — All features implemented and tested

- ✅ Data models and configuration loader
- ✅ Exact duplicate detection (hash-based)
- ✅ Structural duplicate detection (AST-based)
- ✅ Pattern duplicate detection (12 anti-patterns)
- ✅ Metrics calculation and trend analysis
- ✅ Report generation (Markdown + CSV)
- ✅ CLI interface with full argument parsing
- ✅ Git integration (incremental analysis)
- ✅ Heatmap visualization
- ✅ Refactoring suggestion engine