Token Budget Check

Token Budget Check

Analyze markdown files to ensure they comply with token budget guidelines.

Purpose

Knowledge Copilot files have token budgets to optimize AI ingestion. This skill checks files against those budgets and suggests optimizations when over budget.

Token Budgets by Document Type

Procedure

1. Identify Target Files

If a specific file is provided, check that file. Otherwise, scan the directory:

find . -name "*.md" -type f | head -20

2. Calculate Token Estimate

For each file, estimate tokens (words × 1.4):

file="path/to/file.md"
words=$(wc -w < "$file" | tr -d ' ')
tokens=$((words * 14 / 10))
echo "$file: ~$tokens tokens ($words words)"

3. Determine Document Type

Infer type from:

• Filename: SKILL.md → skill, 00-overview.md → overview
• Path: 02-products/ → product doc, 03-ai-enabling/02-profiles/ → agent profile
• Frontmatter: doc_type field if present

4. Compare Against Budget

5. If Over Budget, Analyze Content

Identify optimization opportunities:

• Prose that could be tables (30-50% savings)
• Redundant explanations
• Examples that could be shortened
• Content that belongs in linked docs

Output Format

## Token Budget Report

### Summary
- Files checked: N
- Compliant: N
- Warnings: N
- Over budget: N

### Details

| File | Type | Tokens | Budget | Status |
|------|------|--------|--------|--------|
| path/to/file.md | skill | 1,850 | 2,000 | OK |
| path/to/doc.md | overview | 950 | 800 | OVER (+150) |

### Recommendations

#### path/to/doc.md (950 tokens, budget 800)
- Lines 45-60: Convert feature list to table (-80 tokens)
- Lines 72-85: Remove redundant introduction (-50 tokens)

Efficiency Patterns Reference