# Duplicate Ad Creative Finder > Duplicate Ad Creative Finder - Skill: `raneqbarber/duplicate-ad-creative-finder` (Agent Skill, multi-file: 3 files) - Install (CLI): `npx skillmds@latest add raneqbarber/duplicate-ad-creative-finder` - Raw SKILL.md: https://api.skillmd.com/api/skills/raneqbarber/duplicate-ad-creative-finder/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: raneqbarber (https://skillmd.com/u/raneqbarber) - Updated: 2026-09-22 - Page: https://skillmd.com/skills/raneqbarber/duplicate-ad-creative-finder --- # Duplicate Ad Creative Finder Collapses a list of platform ads down to the distinct creatives actually behind them. Meta and LinkedIn mint a fresh ad ID every time the same image is pointed at a new campaign, audience or placement. Reporting on ad IDs therefore tells you which *placements* ran, never which *creative* worked. On a real B2B library, 2,824 creative links resolved to 1,006 unique ads and only **202 distinct creatives** — a 5× reduction. One image was running under 17 different ad IDs across both platforms. Run this before tagging and before any creative analysis. Everything downstream depends on it. --- ## 1 · Operating contract - **Group by pixels, never by metadata.** Filenames, ad names and campaign names are not identity. The same creative is routinely renamed; different creatives are routinely named the same. - **Never merge without producing the audit.** Every merge decision must be checkable by a human looking at two thumbnails side by side. A silent merge is an unfalsifiable claim. - **Never partially update the outputs.** The four tables are generated together and replaced together. Asset IDs are sequential; a partial update silently desynchronises them. - **Never estimate a hash.** Fingerprinting requires actual computation. Where the runtime cannot execute code, emit the procedure and the file list and say the step was not run. - **Preserve every original ad ID.** Deduplication reduces the *unit of analysis*, never the data. Every ad ID must remain traceable to its creative. --- ## 2 · When to run this Run at the start of any creative analysis project, whenever new creative is added to a library, and before `ad-creative-tagger` — always before, never after. Do not re-run it when new *performance* data arrives for creative already fingerprinted. Identity is a property of the creative, not the reporting period. --- ## 3 · Required inputs and preflight ### Must have | Input | Notes | |---|---| | `ad_id` | One row per platform ad | | `creative_url` | A resolvable link to the actual file — image, video, or every carousel card | | `platform` | Same creative often runs on several; identity is cross-platform | ### Improves the output Ad name, campaign, placement, launch date, and the caption text — which becomes `copy_id` and makes `copy-vs-visual-impact-analyzer` possible later. ### Preflight 1. **URLs resolvable.** Sample 20 links and confirm they fetch. *Halt if a systematic pattern fails* — see §5 on case-mangled IDs, which silently kills half a dataset. 2. **Carousel completeness.** Confirm every carousel has all its cards, in order. *Halt on partial decks.* 3. **Media types identified.** Confirm each row is classified image, video or carousel before hashing; the three take different treatments. 4. **Storage available.** Confirm a cache location exists. Without one the run is not repeatable and every re-run re-downloads everything. --- ## 4 · The procedure ### Step 1 — Repair the source links Before downloading anything, check for systematically broken IDs. The canonical failure: an export arrives with lowercased file IDs. Drive IDs are case-sensitive, so every one of them 404s, and if you are not watching, half the dataset silently drops out. Rebuild the true casing by scraping the folder listing and matching on the lowercased form. Report how many links were repaired. A large number means the export path needs fixing upstream. ### Step 2 — Download and cache Fetch every file in parallel — 16 threads is a reasonable default — into a content cache keyed by a hash of the URL. Convert share links to direct-download endpoints, and handle virus-scan confirmation tokens where the host uses them. Nothing is ever downloaded twice. This is what makes the run repeatable in seconds rather than hours. ### Step 3 — Fingerprint the pixels | Media | Fingerprint | |---|---| | **Image** | One perceptual hash. 64-bit pHash or dHash for grouping; 256-bit where precision matters | | **Video** | Three keyframe hashes at 10%, 50% and 90% of runtime, plus duration | | **Carousel** | One hash per card, retained as an ordered list | Cache the fingerprints separately from the media. Re-running the grouping logic should then cost seconds, not a re-download. ### Step 4 — Group by visual identity - **Images:** group where Hamming distance ≤ 6 on a 64-bit hash. Tighten to ≤ 4 if the library contains many near-identical variants that must stay distinct. - **Video:** require **all three** keyframe hashes to match, and durations to agree within a small tolerance. One frame collides too easily — title cards and fades are near-identical across unrelated ads. - **Carousels:** match each card first, then treat the carousel's identity as the **ordered sequence of its card IDs**. Two carousels match only if every card matches in the same order. A reordered deck is a different creative. ### Step 5 — Mint stable IDs Give each surviving creative a stable, human-readable ID: `IMG-1001`, `VID-1001`, `CAR-1001`, with carousel cards as `CAR-1001-1 … -N` pointing at their parent. These IDs are permanent. Once assigned, an ID is never reused or renumbered, because it will end up in spreadsheets, briefs and other people's decks. ### Step 6 — Pick the surviving URL A creative usually has several URLs pointing at it. Choose one canonical file, preferring the copy you control over a CDN copy that may expire. Record which URLs were merged into it. ### Step 7 — Extract copy identity Where captions are supplied, deduplicate them the same way — normalize whitespace, casing and punctuation, then hash — and assign `copy_id`. This produces the four-level model: ``` asset_id one unique visual IMG-/VID-/CAR-#### copy_id one unique caption COPY-#### creative_id one unique (asset + copy) pair CRV-#### ad_id one platform ad ``` The relationship is genuinely many-to-many, and keeping all four levels is what makes matched-pair analysis possible later. ### Step 8 — Build the audit Produce a row per original ad showing the **assigned creative's thumbnail beside that ad's own original file**. Any row where the two images differ is a bad match. To audit fast: filter to merged duplicates only. Those are the only rows where a match can be wrong — representative files are true by definition. On a 1,006-ad run that is 806 rows to check, not 1,006. --- ## 5 · Computation rules **Non-negotiable.** - **Fan results back out.** Tag and analyze at creative level, then map results back to every ad ID. Never discard ad IDs. - **Hash the pixels, not the file bytes.** Byte hashes differ for re-encodes of the same image and are useless here. - **Never group across media types.** A video and its thumbnail are not the same creative. - **Case sensitivity is not a detail.** File IDs from most cloud hosts are case-sensitive; any pipeline that lowercases them is silently destroying rows. - **Regenerate all four outputs together**, always. --- ## 6 · Output contract Four tables, generated together. | Table | Rows | Answers | |---|---|---| | `creative_inventory` | one per unique creative | The library. Aspect ratio, duration, card count, canonical URL | | `ad_mapper` | one per ad | Given an ad ID, which creative is it | | `asset_rollup` | one per creative | Given a creative, how many ads used it and which — sorted by reuse, so the workhorses sit on top | | `creative_audit` | one per ad | The proof. Assigned thumbnail beside the ad's own original | Plus a run summary: ``` DEDUPLICATION · Jan–Aug 2026 · Meta + LinkedIn source rows 2,824 unique ads 1,006 unique creatives 202 → 5.0× reduction links repaired 506 (case-mangled Drive IDs) most reused IMG-1043 · 17 ads across both platforms audit rows to check 806 (merged duplicates only) ``` --- ## 7 · Failure and degraded states | Situation | What to do | |---|---| | No code execution available | Emit the procedure, the file list and the exact hashing parameters. State plainly that fingerprinting was not run and no grouping is claimed. | | Systematic 404s | Halt at Step 1. Report the pattern — it is almost always a casing or permissions problem, not missing files. | | Partial carousel | Exclude that ad from grouping, list it in `incomplete`. Never hash a partial deck. | | Two creatives group that should not | Tighten the Hamming threshold and re-run. Record the threshold used in the run summary. | | Video keyframes disagree | Leave them separate. False splits are recoverable; false merges corrupt everything downstream. | | No cache location | Warn that the run is not repeatable, then proceed. | --- ## 8 · Worked example **Input:** 2,824 rows exported from Meta and LinkedIn, each with a creative link. **Step 1** finds 506 Meta rows carrying lowercased Drive file IDs. All 506 would have 404'd, dropping the entire Meta half of the dataset. Casing restored by matching against the folder listing. **Step 3–4** fingerprints 2,824 files. Images group at Hamming ≤ 6. One carousel pair matches on five of six cards — **left separate**, because a false merge is unrecoverable. **Step 5** yields 202 creatives: 148 images, 31 videos, 23 carousels. **Step 8** produces 806 audit rows. Spot-checking finds two bad matches, both near-identical seasonal variants of one template. The threshold tightens to ≤ 4 and the run is repeated — from cached fingerprints, in seconds. **Result:** reporting moves from 1,006 ad rows to 202 creative rows. Tagging cost drops by 80%, and every creative now carries one consistent set of tags instead of a dozen slightly different ones. --- ## 9 · Self-check 1. Were links repaired before download, and is the repair count reported? 2. Was grouping done on perceptual hashes, never on names or byte hashes? 3. Do carousels match on the ordered sequence of every card? 4. Is every original ad ID still traceable to its creative? 5. Were all four tables regenerated together? 6. Does the audit table show the assigned thumbnail beside each ad's own original? 7. Is the Hamming threshold recorded in the run summary? --- *MIT licensed.*