Progressive Metadata

Progressive Metadata Prompting Pattern

Purpose

Demonstrate and teach the progressive metadata pattern - using YAML frontmatter in SKILL.md files to achieve significant token reduction (84%) while maintaining full functionality through progressive disclosure.

When to Use

• Creating new skills that need token efficiency
• Building domain expertise skills (PDF extraction, data transformation, protocols)
• Implementing repetitive specialized tasks
• Requiring consistent output formats
• Teaching others how to structure skills effectively

Core Pattern

The progressive metadata pattern uses YAML frontmatter to provide Claude with:

1. Metadata first: Lightweight name and description (~200-1024 tokens)
2. Progressive loading: Full instructions loaded only when skill matches
3. Supporting files: Additional context loaded on-demand via references

Token Savings

Traditional approach (manual instructions):

• ~8,750 tokens per invocation
• Full context repeated every time
• Inconsistent formatting
• High cost per call (~$0.045)

Progressive metadata approach:

• ~1,400 tokens per invocation
• Metadata loaded first, full content only if needed
• Consistent, deterministic outputs
• 84% cost reduction (~$0.007 per call)

Implementation Instructions

Step 1: Create SKILL.md with Frontmatter

---
name: your-skill-name
description: Comprehensive description including what the skill does, when to use it, and trigger keywords. This is loaded first and determines if the skill activates.
allowed-tools: Read, Write, Grep  # Optional
---

# Your Skill Name

[Full instructions here - loaded only if skill activates]

Step 2: Write Comprehensive Description

The description field is critical for progressive disclosure:

Must include:

• WHAT the skill does (action verbs)
• WHEN to use it (specific scenarios)
• TRIGGERS (keywords users would mention)
• CONTEXT (domain terms, file types)

Example:

description: Extract structured form fields from PDF documents with validation. Use when working with PDF forms, invoices, tax documents, or any document requiring data extraction. Handles OCR, field detection, and schema validation. Supports .pdf files, form field extraction, and structured data output. Triggers on "PDF extraction", "parse PDF", "invoice data", "form fields".

Step 3: Structure Instructions

After frontmatter, organize instructions clearly:

## Purpose
[Brief explanation]

## When to Use
[Specific scenarios]

## Core Instructions
1. Step-by-step process
2. Decision trees for complex logic
3. Examples showing patterns
4. Error handling guidance

## Examples
[Code or usage examples]

## Dependencies
[Required tools or libraries]

Step 4: Add Supporting Files (Optional)

For complex skills, use progressive disclosure with supporting files:

.claude/skills/your-skill/
├── SKILL.md (always loaded with metadata)
├── examples/
│   └── advanced-usage.md (loaded on reference)
└── templates/
    └── template.yaml (loaded on reference)

Reference from SKILL.md:

For advanced examples, see [examples/advanced-usage.md](examples/advanced-usage.md).

Claude loads these files only when the link is followed.

Real-World Example: PDF Field Extractor

This example demonstrates the full pattern:

SKILL.md frontmatter:

---
name: pdf-field-extractor
description: Extract structured form fields from PDF documents with type detection and validation. Use when parsing invoices, tax forms, medical records, or any PDF with form fields. Handles OCR quality issues, spatial relationships, and multi-page forms. Outputs JSON with confidence scores. Triggers on "extract PDF", "PDF fields", "invoice parsing", "form data extraction".
---

SKILL.md body:

# PDF Field Extractor

## Purpose
Extract structured data from PDF form fields with validation.

## When to Use
- Invoice processing
- Tax form data extraction
- Medical records parsing
- Any PDF with labeled fields

## Core Instructions

### 1. Field Detection
- Identify labeled fields (e.g., "Name:", "Date:", "Amount:")
- Detect field types (text, number, date, checkbox)
- Extract spatial relationships
- Validate against schema if provided

### 2. Output Format
```json
{
  "document_metadata": {
    "filename": "invoice_001.pdf",
    "pages": 2
  },
  "fields": [
    {
      "label": "Invoice Number",
      "value": "INV-2025-001",
      "type": "text",
      "confidence": 0.98,
      "page": 1
    }
  ],
  "validation": {
    "required_fields_present": true,
    "format_errors": []
  }
}

3. Validation Rules

• Dates: ISO 8601 format
• Currency: Non-negative decimal
• Email: RFC 5322 format
• Low confidence (<0.7): Flag for review

Examples

See examples/invoice-extraction.md for detailed examples.

Dependencies

• PDF with text layer or OCR
• Optional: pdftotext for pre-processing

**examples/invoice-extraction.md:**
```markdown
# Invoice Extraction Example

## Before (Manual Instructions)
[8,750 tokens of detailed instructions repeated every time]

## After (With Progressive Metadata)
[1,400 tokens - metadata + brief task description]

## Performance Comparison
| Metric | Without Skill | With Skill | Improvement |
|--------|--------------|------------|-------------|
| Tokens | 8,750 | 1,400 | 84% reduction |
| Cost | $0.045 | $0.007 | 84% savings |
| Time | ~8 sec | ~3 sec | 62% faster |

Performance Metrics

Based on internal Anthropic data (Oct 2025):

Best Practices

Do:

• Write comprehensive descriptions (200-1024 chars)
• Include specific trigger keywords
• Use clear step-by-step instructions
• Provide concrete examples
• Reference supporting files for progressive disclosure
• Keep SKILL.md focused on core logic

Don't:

• Write vague descriptions ("helps with files")
• Omit trigger keywords from description
• Bloat SKILL.md with edge cases (use supporting files)
• Duplicate information across files
• Include non-standard frontmatter fields

Use Cases

Best for:

• Repetitive specialized tasks (PDF extraction, data transformation)
• Domain expertise (legal, medical, engineering protocols)
• Consistent output requirements
• High-volume operations

Not suitable for:

• One-off exploratory tasks
• Rapidly changing requirements
• Tasks needing full context flexibility

Version

v1.0.0 (2025-10-23) - Based on official Claude Agent Skills specification