Data Flywheel

Data Flywheel - approved runs into reusable intelligence

Use this skill when a user wants to turn repeated human-approved agent work across Claude Code, Hermes, OpenClaw, Codex, Cursor, or custom .agent/ loops into local artifacts for retrieval, evals, prompt shrinking, and optional future open-weight model/adapters.

The flywheel is:

approved run
-> redacted trace
-> context card
-> eval case
-> training-ready JSONL
-> optional downstream SLM/adapter experiment later

This skill creates the harness. It does not train a model.

Hard Rules

• Use only human-approved runs. Rejected or unknown-review runs can become failure-mode notes, not trainable examples.
• Redaction must pass before anything is marked trainable.
• Do not store raw prompts, raw code, client names, addresses, phone numbers, emails, secrets, credentials, or unredacted CRM records.
• Keep .agent/flywheel/ private and gitignored unless the user explicitly commits sanitized examples.
• Stay model-agnostic. Mention model families only as downstream examples.

Inputs

Default local input:

.agent/flywheel/approved-runs.jsonl

Each line should be a sanitized run record with:

• domain
• workflow
• harness
• instruction
• input_redacted
• output_approved
• human_review.status as accepted or edited
• redaction_status: passed
• pii_level
• optional stable_rules, tool_contracts, eval_tags, failure_modes

Export

Run:

python3 .agent/tools/data_flywheel_export.py

Outputs go to:

.agent/flywheel/exports/<YYYY-MM-DD>/

Key outputs:

• trace-records.jsonl
• training-examples.jsonl
• eval-cases.jsonl
• context-cards/<domain>/<workflow>.md
• context-cards/<domain>/<workflow>.json
• flywheel-metrics.json

Readiness Checks

Use these as heuristics, not hard rules:

• 10-25 approved runs: useful first context card
• 25-100 approved runs: first eval set and repeated failure modes
• 100-300 approved runs: context compression and routing measurement
• 500-1,500 high-quality examples: narrow adapter experiment candidate
• 2,000-10,000+ examples: broader workflow-family corpus

What To Report

When finishing, report:

• traces exported
• trainable examples exported
• eval cases exported
• context cards created
• redaction pass rate
• acceptance rate by workflow
• workflows that should stay frontier-model/manual-review
• workflows that may become SLM/adapter candidates later

Self-rewrite hook

If users repeatedly ask for the same domain-specific fields, add them to a local context card or schema example instead of hard-coding them into this general skill.