Skilled Agent V500

Skilled Agent Architecture (v5.0.0)

Overview

The Problem

Three A/B experiments (Feb 21-27, 2026) conclusively showed the multi-agent system (v4.5.0) provided zero benefit:

Root cause: Agents had no genuine skills. They were generic LLM prompt-responders that observed 6 scalar metrics and output JSON. Literature review (EnvGen, ATLAS, QuantAgent, LangProp, MLE-Dojo) showed skilled agents require: callable tools, simulate-before-apply, adaptive prompt evolution, fewer but higher-quality consultations.

Verified Workflow

Architecture

NativePPOTrainer.train(validation_callback)
      |
      v  (~15 validations per run)
SkilledTrainer
      |
      +---> Rule-Based Autopilot (handles 11-12 of 15 validations)
      |      - Proactive overtrading/collapse fix (existing, kept)
      |      - Auto-best checkpoint (existing, kept)
      |      - High-confidence instincts (auto-applied, no LLM)
      |
      +---> Skilled Agent (~4 consultations at phase transitions)
               |
               v
          claude_agent_sdk.query(prompt, options)
               |
               +---> @tool simulate_reward_change    (~0ms)
               +---> @tool get_regime_analysis       (~100ms)
               +---> @tool run_mini_backtest          (~5-10s)
               +---> @tool query_knowledge_base       (~10ms)
               +---> @tool estimate_pnl_impact        (~10ms)
               |
               v
          Parse recommendation -> AgentSafetyWrapper -> _apply_action()
               |
               v  (next validation)
          Verify predicted vs actual -> Update instinct confidence

Consultation Triggers (4 event-based, NOT interval-based)

1. EARLY→MID phase (~15% progress): Agent reviews early metrics, suggests initial weight adjustments
2. MID→LATE phase (~70% progress): Agent reviews convergence, suggests final tuning
3. First pathology: Overtrading, direction collapse, or reward collapse detected
4. Training completion: Post-mortem — which instincts to create/update?

Simulate-Verify Loop

1. Simulate: Agent calls simulate_reward_change + estimate_pnl_impact → evidence-based recommendation
2. Apply: Standard _apply_action() with same bounds/gates as multi-agent
3. Verify (next validation): Direction correct + magnitude within 2x = confirmed
4. Learn: Confirmed instincts gain confidence. At 5+ confirmations ≥0.8 → graduate to autopilot rule

Prompt Evolution

8 learnable parameters (numeric + categorical):

• overtrading_threshold (0.20), direction_collapse_threshold (0.40)
• reward_adjustment_magnitude (0.03), early_phase_cutoff (15.0)
• tool_budget (3), primary_objective ("P&L")
• risk_tolerance ("moderate"), preferred_pattern ("increase_slippage_for_overtrading")

After each run, records (parameter_value, fitness_delta). After 3+ data points, nudges numeric values 20% toward historically best-performing value. Pure numerical optimization — no LLM.

Failed Attempts

Final Parameters

SkilledAgentConfig

SkilledAgentConfig(
    max_consultations=4,           # Budget per run
    model="claude-sonnet-4-5",     # Cost-effective for tool use
    phase_early_to_mid_pct=15.0,   # EARLY→MID trigger
    phase_mid_to_late_pct=70.0,    # MID→LATE trigger
    proactive_low_hold_threshold=2, # Consecutive low-hold before auto-fix
    proactive_collapse_threshold=2,  # Consecutive collapse before auto-fix
)

Cost

Safety (Unchanged)

All safety bounds from AgentSafetyWrapper preserved:

• ±0.05/call, ±0.15 cumulative, 0.01 floor per reward component
• Phase gate: No reward changes before 15% progress
• Tiered fitness decline gate (strict for rollback/halt, moderate for weight changes)
• Forbidden actions: bypass_all_gates, disable_risk_controls, etc.
• Rate limiting: 50 actions/hour, 60s minimum interval

Key Insights

1. Tool-augmented > prompt-only: Giving the agent callable tools (simulate, backtest, knowledge search) produces evidence-based recommendations instead of generic advice
2. Fewer, better consultations: 4 event-based calls beat 25-33 interval-based calls in both quality and cost
3. Simulate-verify is essential: Without ground truth checking, agents accumulate false confidence
4. Prompt evolution works without LLM: Simple numerical optimization (20% nudge toward best) outperforms meta-prompting
5. Autopilot graduation: High-confidence instincts (5+ confirmations ≥0.8) become rules — system gets faster over time
6. Safety wrapper is reusable: AgentSafetyWrapper works identically for single-agent and multi-agent

References

• EnvGen (arXiv:2307.01548): Environment generation for skill learning
• ATLAS (arXiv:2407.00813): Tool-augmented agents for science
• QuantAgent (arXiv:2402.11412): Self-improving quantitative agents
• LangProp (arXiv:2312.09473): Code optimization via LLM agents
• MLE-Dojo (arXiv:2503.07475): Interactive learning environments for agents
• Adaptive-OPRO / Symbolic Learning: Prompt parameters as learnable values

Dependency

Requires claude-agent-sdk>=0.1.44 (added to config/requirements.txt). The system gracefully degrades when the SDK is not installed — AGENT_SDK_AVAILABLE guard disables consultations and falls back to rule-based autopilot only.