Deduplicate Near Identical Chunks

Use when the index returns five copies of the same paragraph from boilerplate or mirrored docs. Detects near-duplicate chunks so one version is retrieved and the rest collapse.

jiayaoqijia Updated 81 repo stars

File contents

jiayaoqijia/cryptoskill commit 8f376ea925

Frequently asked questions

npx get-skill add jiayaoqijia/deduplicate-near-identical-chunks