# Token Squeeze > Deterministic no-LLM compressor for text that will be REUSED — saved prompts, system prompts, skill bodies, repeatedly-pasted docs. Meaning-leaning, protects code/URLs/paths/quotes/numbers/negations. NOT for live chat turns: those tokens are already spent; this pays only when the compressed text replaces the original in future contexts. Use when: "compress this prompt/doc for reuse", "shrink this system prompt", "token-squeeze". Bundled Node CLI, no API key. - Skill: `evan-daruwalla/token-squeeze` (Agent Skill, multi-file: 14 files) - Install (CLI): `npx skillmds@latest add evan-daruwalla/token-squeeze` - Raw SKILL.md: https://api.skillmd.com/api/skills/evan-daruwalla/token-squeeze/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: Evan-Daruwalla (https://skillmd.com/u/evan-daruwalla) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/evan-daruwalla/token-squeeze --- This skill wraps a deterministic CLI. It does NOT ask you to compress text by hand — invoke the tool so results are reproducible and guard-checked. WHEN IT PAYS (state this if the user aims it at live chat). Compressing text already in the context window saves nothing — those tokens were billed on arrival. token-squeeze only nets tokens when its output is reused: a saved prompt, a system prompt, a skill body, a doc pasted across many sessions. For a one-off message, skip it. RUN IT. From the skill directory: `npm install` once (pulls gpt-tokenizer), then: - `node cli.js ` — compressed text to stdout. - `node cli.js --stats` — adds a token-before/after + guard line on stderr. - `node cli.js --json` — structured {tokensBefore, tokensAfter, savedPct, subs, subsDetail, guard}; subsDetail is the per-substitution ledger ("what changed"), show it when the user asks what was done. - `node cli.js - --stats` — read from stdin. - add `--clean` only for text carrying junk whitespace (pasted logs, tables); it's off by default because it saves ~0% on normal prose. Exit code is non-zero if the guard trips, so it's safe to script. WHAT IT DOES. Layer B (default): token-aware dictionary substitution — "in order to"->"to", "due to the fact that"->"because", "utilize"->"use", plus deletion of pure filler. Each dictionary entry is self-filtered at load against the tokenizer and dropped unless it strictly reduces tokens, so a bad entry can't make output worse. Layer A (`--clean`, opt-in): lossless whitespace reclamation. GUARANTEES AND LIMITS (report honestly, don't oversell). - Measured on the bundled corpus: ~25% overall, 37-42% on verbose prose, 13-17% on technical text, 0% on already-tight prompts (a tight prompt returns byte-identical). Run `node test.js` to reproduce. - Protected spans are never altered: fenced/inline code, URLs, Windows paths, double-quoted strings, dotted identifiers (auth.js, example.com, decimals). A five-way guard verifies every run and fails loudly: numbers; negation/constraint words in ANY case (no/not/never/must/cannot + contractions); dotted identifiers; protected spans; idempotence (running the output through again must be byte-identical). - Homonym-risky words are deliberately NOT in the dictionary (e.g. "leverage" — financial noun; "interface with" — UI noun phrase); a deterministic tool can't disambiguate, so it doesn't try. Same for phrases whose compression would delete a negation ("there is no doubt that") — the guard outranks the saving. - Token counts use the o200k (tiktoken) tokenizer as a PROXY for Claude's tokenizer, which Anthropic does not publish. Real Claude savings will be close but not identical. For exact numbers, count via the Anthropic count-tokens API (needs an API key). - Meaning-leaning, not string-lossless: substitutions preserve propositional content but change wording; emphasis from deleted filler is lost by design. Do not use where exact wording is contractual (legal text, quoted material) — those cases are protected only if quoted. After running, show the user the savedPct and the guard result; if the guard fails, surface it rather than returning the output silently. REPORTING FORMAT. Show before/after with the savedPct, then two short lists: **Preserved** (protected spans, negations, numbers — what the guard confirmed survived) and **Dropped** (pure-filler phrases removed entirely, not substituted — from `subsDetail`, the "-> (deleted)" entries). Never silently swap the user's text; the diff and the guard result are always shown. FOR LONG REUSABLE PROMPTS (~200+ words). Beyond running the dictionary substitution, restructure the result as: **INTENT** (one sentence — what the prompt is for), **CONSTRAINTS** (bullets — the non-negotiable rules), **CONTEXT** (only what the model can't infer on its own). This is a manual restructuring step on top of the deterministic pass, not something the CLI does — do it only when asked to prep a prompt for reuse (a saved prompt, a system prompt, a skill body), not for a one-off message.