Detect Duplicates

Detect Duplicate Feature Requests

You are helping a WooCommerce Marketplace team member identify and act on duplicate feature requests for a specific product.

Read first:

• .claude/skills/shared/RULES.md
• .claude/skills/shared/DISPLAY.md
• .claude/skills/shared/PHASE_LOOP.md

Step 1 — Resolve the product

Follow .claude/skills/shared/RESOLVE_PRODUCT.md.

Step 2 — Fetch all open feature requests

Skip this step if invoked by the orchestrator. The orchestrator passes the FRs in input_path (a JSONL file already on disk). Use it directly as the input to the detection script in Step 3 and skip writing /tmp/frs-<id>.jsonl.

Call wccom-feature-requests-list with per_page=100 and status: "publish". Paginate. Collect id, title, description, status, votes, date, url.

If the first page returns 0 results, stop and report: "No open feature requests found for this product."

Step 2b — Choose comparison depth

Skip this step if invoked by the orchestrator. The orchestrator passes comparison_mode (fast or in-depth).

Ask:

How thorough should the duplicate scan be?

  [F] Fast       — title-based matching only (script + false-positive pruning).
                   Good for quick cleanup. Misses duplicates with different titles
                   but similar descriptions.
  [D] In-depth   — title + full description comparison. Claude reads every
                   description to catch hidden duplicates.
                   More thorough, takes longer.

Reply F or D.

Store as comparison_mode (fast or in-depth). Default to fast.

Step 2c — Write FRs to a temp file

Skip this step if invoked by the orchestrator. Use the orchestrator's input_path directly.

Use Write to create $TMPDIR/frs-<product_id>.jsonl. One JSON object per line: id, title, description, votes, date, url.

Step 3 — DETECTION STARTS HERE — Run the duplicate detection script

(Orchestrator subagent: begin reading from this step. Use <input_path> wherever the script command shows $TMPDIR/frs-<product_id>.jsonl.)

Choose the command based on comparison_mode:

• fast: python3 scripts/detect_duplicates.py $TMPDIR/frs-<product_id>.jsonl --json
• in-depth: .venv/bin/python3 scripts/detect_duplicates.py $TMPDIR/frs-<product_id>.jsonl --semantic --rerank --json

The --semantic --rerank flags require sentence-transformers in .venv/. If import error, fall back to fast and note it.

Parse the JSON output:

{
  "groups": [
    {
      "confidence": "high" | "low",
      "max_similarity": 0.75,
      "primary": {"id": 123, "title": "...", "votes": 5, "date": "...", "url": "..."},
      "duplicates": [{"id": 456, "title": "...", "votes": 2, "date": "...", "url": "..."}]
    }
  ],
  "borderline_pairs": [
    {"similarity": 0.58, "fr_a": {...}, "fr_b": {...}}
  ]
}

Normalise script confidence (high/low lowercase) to High/Low per RULES.md. Store groups as script groups and borderline_pairs for Step 3b-ii. If the script fails entirely, fall back to Step 3b and note it.

Step 3b — Claude semantic review

3b-i — Review script groups for false positives

For each group, read full titles and descriptions of every member. If a pair is clearly NOT the same ask (same keyword but different intent), remove the false-positive member. If a group reduces to one member, discard it.

Freed FRs go to a re-evaluation pool — they may be true duplicates of a different FR that the script missed.

Do not discard a group for being support/spam/off-topic. Duplicates are duplicates regardless of quality. Surface them here.

Upgrade or downgrade confidence:

• Upgrade to High if full text confirms a match beyond title similarity.
• Downgrade to Low if the match is only superficial.

3b-ii — Semantic review of borderline pairs

Skip entirely if comparison_mode = fast. Proceed to 3b-iii.

If in-depth:

Review two inputs:

1. borderline_pairs from the script — pairs near the threshold.
2. re-evaluation pool from 3b-i.

For each borderline pair, read both title and description in full. Decide: same core ask? If yes, add as a new group. Be conservative.

For pool FRs not covered by any pair, pay attention to vague short titles ("Booking", "Hi :)") — read full descriptions before comparing.

Assign confidence (High / Low). Primary = most votes, then oldest if tied. Add as new groups.

3b-iii — Merge

Combine script groups (after pruning) with new semantic groups. Sort: High first, then by primary vote count descending.

Step 4 — Present the report

Pagination — grouped phase, see DISPLAY.md. Decode HTML entities. Translate non-English excerpts per RULES.md.

For each group:

## Group [N]: [short description of the shared ask]
Confidence: High / Low
Reason: [one sentence — what makes these the same ask]

✅ Keep:  ID [id] — "[title]" ([votes] votes · [status] · [date])
          [url]
          "[description — first 2–3 sentences, trimmed if long]"
❌ Close: ID [id] — "[title]" ([votes] votes · [status] · [date])
          [url]
          "[description — first 2–3 sentences, trimmed if long]"

End with: X requests scanned · Y duplicate groups · Z requests could be closed.

Flagged record schema (for orchestrator output JSONL)

One record per duplicate-to-close:

{ "id": <int>, "title": "<str>", "url": "<str>",
  "reason": "<one sentence>", "confidence": "High" | "Low",
  "excerpt": "<first 1–3 sentences>",
  "votes": <int>, "date": "<YYYY-MM-DD>",
  "group_id": <int>,
  "primary": { "id": <int>, "title": "<str>", "url": "<str>",
               "status": "<str>", "votes": <int>, "date": "<YYYY-MM-DD>",
               "excerpt": "<first 1–3 sentences>" } }

Step 5 — Confirmation menu

List every closeable duplicate grouped by group number. Within each group, sort High → Low. Assign sequential numbers across all groups (stable for pagination).

Which requests should I close?

  Group 1: [short group label]
    [1] ID [id] — "[title]" [High]
    [2] ID [id] — "[title]" [High]
    [3] ID [id] — "[title]" [Low]

  Group 2: [short group label]
    [4] ID [id] — "[title]" [High]
  ...
  [A] All of the above
  [H] High confidence only      ← omit if no High items (see RULES.md)
  [N] None / skip

Reply with numbers (e.g. "1 3"), A for all, H for High only, or N to skip.

Wait for the user's reply. Then group approved requests by their comment template (same primary + same primary status = same template). For each distinct template, show the preview once with the list of FRs:

📝 Comment preview — applies to [N] request(s):
  • ID [id] — "[title]"
  • ID [id] — "[title]"

---
[full comment text, with [primary_url] filled in]
---

After all previews: "Post these comments and close the requests? (Y / N)" If N, abort.

Then for each approved duplicate, look up its group's primary and process in this order:

1. Post closing comment via wccom-feature-requests-comment. Template depends on primary status:

   Primary open with more votes:

   Hi there,
   
   Thank you for your suggestion!
   
   We noticed that there is a similar request about this, which currently has more votes: [primary_url]. To better prioritize requests, we aim to keep 1 request open per feature. That's why I will close this one. I encourage you to upvote the above request instead.
   

   Primary open with equal votes (kept because older):

   Hi there,
   
   Thank you for your suggestion!
   
   We noticed that there is a similar request about this that has been open longer: [primary_url]. To better prioritize requests, we aim to keep 1 request open per feature. That's why I will close this one. I encourage you to upvote the above request instead.
   

   Primary planned or in-progress:

   Hi there,
   
   Thank you for your suggestion!
   
   We noticed that this feature is already planned and being tracked here: [primary_url]. That's why I will close this duplicate. You can follow progress on that request instead.
   

   Primary completed:

   Hi there,
   
   Thank you for your suggestion!
   
   Great news — this feature has already been implemented! You can find more details here: [primary_url]. That's why I will close this request.
   

2. Update status via wccom-feature-requests-update-status → "closed".

3. Write failure handling — on failure, do not record as actioned; report in summary; continue. If comment posted but status update failed, note the half-completed state.

Step 6 — Extract insights for each primary

After closing approved duplicates, for each group with at least one closure, compare full descriptions of every closed request against the primary's. Look for:

• Additional use cases or scenarios not covered by the primary
• Different user workflows or perspectives
• Concrete details, examples, or edge cases
• Explicit requirements/constraints mentioned only in closed requests

If unique insights are found, draft a comment for the primary. Plain text only (see RULES.md). Example:

Hi there,

We recently consolidated a few related requests into this one. In doing so,
we noticed some additional use cases and perspectives worth capturing here:

- [insight 1]
- [insight 2]
...

We've noted these for the team when reviewing this request.

Preview:

📝 Draft comment for primary — ID [primary_id] — "[primary_title]"
[primary_url]

---
[full draft comment]
---

Post this comment to the primary request? (Y / N)

If Y: call wccom-feature-requests-comment on the primary's ID. Do not update the primary's status.

If no unique insights across any closed duplicate: "No additional insights found — primary requests are already comprehensive." Skip the step.

Step 7 — Confirmation summary

Done.
❌ Closed: ID [id] — "[title]"
   [closed_url]
✅ Primary: ID [id] — "[title]" ([status])
   [primary_url]

⚠️  Failed: ID [id] — "[title]" — [error]

Then follow PHASE_LOOP.md. Additionally, the standalone re-scan prompt accepts a group-number reference:

Need more context on a group? Type the group number (e.g. "Group 3").

If the user specifies a group number: show full title, description, votes, date, url for every request in that group (primary + all closeable duplicates), then re-ask the loop question.