ariffazil FORGE-data-compression
Purpose
Long logs, massive codebases, and multi-organ evidence bundles routinely exceed model context windows. This skill compresses input through a staged pipeline while maintaining ≥0.95 semantic fidelity.
Pipeline
- Chunk — Split input into semantic units (log blocks, code functions, evidence sections)
- Score — Rank chunks by relevance to the query/task context
- Compress — Apply lossy compression to low-relevance chunks, lossless to high-relevance
- Reconstruct — Reassemble into compressed artifact with provenance markers
- Verify — F2 TRUTH check: decompress sample and compare against original
Compression Modes
log-compress: Strip timestamps, deduplicate repeated lines, extract error/warn signalscode-compress: Collapse boilerplate, preserve signatures + logic + commentsevidence-compress: Rank by epistemic rung, compress low-confidence sectionsstreaming: Chunk-summarize-recurse until target token count reached
Floors
- F2 TRUTH: ≥ 0.95 fidelity. Compression must not fabricate or distort meaning.
- F4 CLARITY: Compressed output must be more readable than raw input.
- F7 HUMILITY: Report compression ratio and what was lost.