a2ui-project Inference Format Optimizer
This skill provides procedural workflows, CLI orchestrators, decision guardrails, and subagent protocols for iteratively optimizing A2UI inference formats.
Quick-Start CLI Cheatsheet
All execution scripts live under scripts/ in this skill:
| Action | Executable Command |
|---|---|
| Run Fast Validation Eval | python scripts/optimize_format.py --format <format> |
| Run Full Evaluation Suite | python scripts/optimize_format.py --format <format> --full |
| Test Parsing / Compilation | python scripts/optimize_format.py --format <format> --compile "(Card (Text \"Hi\"))" |
| Compare vs Baseline | python scripts/compare_results.py --baseline eval/iterative_format_optimizer/baselines/<format>/unbounded_run_meta.json eval/iterative_format_optimizer/logs/temp_optimization/ |
| Archive Run Artifacts | python scripts/optimize_format.py --format <format> --archive --hypothesis "..." --status KEEP [--history-dir <path>] |
| Sync Multi-Worktree History | python scripts/sync_history.py [--history-dir <path>] |
Detailed References
- Scoring & Decision Rules: See references/scoring_model.md for $S_{\text{opt}}$ formula, correctness guardrails, and efficiency caps.
- Subagent Worktree Protocol: See references/subagent_protocol.md for launching subagents in isolated git worktrees.
- Subagent Prompt Template: See templates/subagent_prompt.md for launching pass tasks via
invoke_subagent.
The 6-Step Optimization Workflow
- Analyze History: Inspect past runs in
eval/iterative_format_optimizer/history/<format>/and readeval/iterative_format_optimizer/history_summary.mdto avoid repeating past reverted hypotheses. - Implement Hypothesis: Modify
compiler.py,prompt_generator.py, orparser.pyunderagent_sdks/python/a2ui_agent/src/a2ui/inference_formats/experimental/<format>/. - Run Unit Conformance Tests: Verify code changes pass pytest unit tests.
- Execute Benchmark Evaluation: Run
python scripts/optimize_format.py --format <format>. - Evaluate Decision Rules:
- Must pass Pytest and maintain baseline accuracy.
- Code Output Tokens must NOT expand $> +5%$.
- Keep change if composite score $S_{\text{opt}}$ improves; revert otherwise (
git reset --hard HEAD).
- Archive & Synchronize: Archive run with
--archiveand update history index usingpython scripts/sync_history.py.