Token Slim

Token Saver (token-slim)

• Author: Evan Song · github.com/Songhonglei
• Repository: https://github.com/Songhonglei/better-agent-skills
• License: MIT

Helps AI agents and their users reduce token consumption by auditing workspace files, identifying bloat, and guiding targeted cleanup.

Core Concept

Every project-context file (memory index, agent config, heartbeat, etc.) is injected into every single session. Keeping them lean = direct, compounding savings. The goal is a three-tier memory layout:

memory index        ← always loaded, < 150 lines — pointers + essentials only
memoryres/*.md      ← loaded on demand — reference tables, specs, backlogs
memory/YYYY-MM-DD.md ← raw daily log — rarely loaded

For full strategy details: see references/strategies.md

Mode Routing

After matching a mode, first read the corresponding reference file, then follow that flow. Do not rely on memory.

Execution Modes

Confirm mode (default)

Show each proposed change, wait for user approval before modifying.

Batch mode

Triggered when the user says "just do it" / "按你说的来" / "全部做". Execute all changes without per-item confirmation; deliver a unified summary at the end.

Dry-run mode

Preview without modifying any files:

SKILL_DIR="$(dirname "$(readlink -f "$0")")/skills/token-slim" 2>/dev/null \
  || SKILL_DIR=./skills/token-slim
python3 "$SKILL_DIR/scripts/scan_workspace.py" --workspace . --dry-run

Or simply, from the workspace root:

python3 <skill-install-path>/scripts/scan_workspace.py --workspace . --dry-run

🔥 Brutal Mode — Toggle

Triggers:

• Enable: "enable brutal mode" / "开启狂暴模式" / "打开狂暴模式"
• Disable: "disable brutal mode" / "关闭狂暴模式" / "退出狂暴模式"

Enable flow:

1. Check the workspace agent config file for <!-- token-slim:token-habits-start --> anchor
2. If absent → append the full template (incl. brutal mode section — see references/mode-a-onboarding.md Step 8)
3. If present → replace the whole block with the full template
4. Reply: "🔥 Brutal mode enabled — concise outputs, no preamble."

Disable flow:

1. Find the block <!-- token-slim:token-habits-start --> … <!-- token-slim:token-habits-end --> in the agent config
2. Replace the brutal-mode section with just a header placeholder:

   ### 🔥 Brutal Mode (max output efficiency) <!-- Brutal mode: disabled -->
   

3. Reply: "✅ Brutal mode disabled — back to normal."

First-time use (after Mode A Step 8 completes): Proactively offer:

"🔥 Want to enable brutal mode? When on, the agent gives results only — no step-by-step narration. Saves tokens and reading time. Say 'enable brutal mode' to toggle (can be turned off anytime)."

Undo (rollback)

scan_workspace.py itself does not modify files and does not auto-backup.

Before any file modification the agent must:

1. Tell the user which files will be modified
2. Ask "Do you want me to back these up first?"
3. On confirmation, copy the soon-to-be-modified files to a timestamped dir:

   <workspace>/.token-slim/undo-<YYYYMMDD-HHMMSS>/
   

4. Report the backup path so the user can restore later.

See references/mode-a-onboarding.md Step 3 for the canonical recipe.

tiktoken installation

token-slim uses tiktoken for accurate token counting (vs ~40-60% heuristic error). Install attempts run in this order with automatic fallback:

python3 <skill-install-path>/scripts/install_tiktoken.py --workspace .

BPE vocabulary cache is fetched automatically by tiktoken on first encode from the OpenAI public blob (openaipublic.blob.core.windows.net) and stored under <workspace>/.cache/tiktoken/ (CWD-anchored — survives $HOME wipes on container/VM environments where only the working directory is persistent).

Check current state:

python3 <skill-install-path>/scripts/install_tiktoken.py --check --workspace .

scan_workspace.py auto-detects tiktoken at runtime; no manual setup needed after installation.

Dependencies

Required:

• Python 3.8+ (python3 available on PATH)

Optional:

• tiktoken for precise token counting (install via the script above). Without it, the scanner uses a CJK/ASCII heuristic with ~40-60% error.

Everything else uses the Python standard library.

What NOT to move

Never suggest moving:

• Behavioural rules, safety constraints, must/never guidelines
• Cron job configurations and failure protocols
• Active heartbeat tasks and retry queues
• Identity / persona core (soul/identity config content)
• Anything with ⚠️ or explicit "always loaded" markers

When in doubt, ask the user.

Key files

CLI reference

scan_workspace.py

python3 scan_workspace.py [--workspace <path>] [--json] [--dry-run]

install_tiktoken.py

python3 install_tiktoken.py [--check] [--workspace <path>]

⚠️ Skills vs workspace-file tokens

Skills (SKILL.md): each skill injects only its name + description + location per session — around 24 tokens. The full body is only loaded when the agent explicitly reads it. Large SKILL.md files affect call latency, not per-session baseline.

Workspace files (memory index, agent config, etc.): injected verbatim every session — these are the real optimisation target.