# Inference Format Optimizer

> Inference Format Optimizer

- Skill: `a2ui-project/inference-format-optimizer` (Agent Skill, multi-file: 19 files)
- Install (CLI): `npx skillmds@latest add a2ui-project/inference-format-optimizer`
- Raw SKILL.md: https://api.skillmd.com/api/skills/a2ui-project/inference-format-optimizer/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: a2ui-project (https://skillmd.com/u/a2ui-project)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/a2ui-project/inference-format-optimizer

---


# Inference Format Optimizer

This skill provides procedural workflows, CLI orchestrators, decision guardrails, and subagent protocols for iteratively optimizing A2UI inference formats.

---

## Quick-Start CLI Cheatsheet

All execution scripts live under `scripts/` in this skill:

| Action                          | Executable Command                                                                                                                                                                |
| :------------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Run Fast Validation Eval**    | `python scripts/optimize_format.py --format <format>`                                                                                                                             |
| **Run Full Evaluation Suite**   | `python scripts/optimize_format.py --format <format> --full`                                                                                                                      |
| **Test Parsing / Compilation**  | `python scripts/optimize_format.py --format <format> --compile "(Card (Text \"Hi\"))"`                                                                                            |
| **Compare vs Baseline**         | `python scripts/compare_results.py --baseline eval/iterative_format_optimizer/baselines/<format>/unbounded_run_meta.json eval/iterative_format_optimizer/logs/temp_optimization/` |
| **Archive Run Artifacts**       | `python scripts/optimize_format.py --format <format> --archive --hypothesis "..." --status KEEP [--history-dir <path>]`                                                           |
| **Sync Multi-Worktree History** | `python scripts/sync_history.py [--history-dir <path>]`                                                                                                                           |

---

## Detailed References

- **Scoring & Decision Rules**: See [references/scoring_model.md](references/scoring_model.md) for $S_{\text{opt}}$ formula, correctness guardrails, and efficiency caps.
- **Subagent Worktree Protocol**: See [references/subagent_protocol.md](references/subagent_protocol.md) for launching subagents in isolated git worktrees.
- **Subagent Prompt Template**: See [templates/subagent_prompt.md](templates/subagent_prompt.md) for launching pass tasks via `invoke_subagent`.

---

## The 6-Step Optimization Workflow

1. **Analyze History**: Inspect past runs in `eval/iterative_format_optimizer/history/<format>/` and read `eval/iterative_format_optimizer/history_summary.md` to avoid repeating past reverted hypotheses.
2. **Implement Hypothesis**: Modify `compiler.py`, `prompt_generator.py`, or `parser.py` under `agent_sdks/python/a2ui_agent/src/a2ui/inference_formats/experimental/<format>/`.
3. **Run Unit Conformance Tests**: Verify code changes pass pytest unit tests.
4. **Execute Benchmark Evaluation**: Run `python scripts/optimize_format.py --format <format>`.
5. **Evaluate Decision Rules**:
   - Must pass Pytest and maintain baseline accuracy.
   - Code Output Tokens must NOT expand $> +5\%$.
   - Keep change if composite score $S_{\text{opt}}$ improves; revert otherwise (`git reset --hard HEAD`).
6. **Archive & Synchronize**: Archive run with `--archive` and update history index using `python scripts/sync_history.py`.