Agent Validation V420

Agent Validation v4.2.0

Context

Three agent validation experiments were run (Jan 31, Feb 18, Feb 19 2026), burning $150+ in GPU time with zero useful results. Root cause: agents had only ONE lever (entropy), LR adjustment was a confirmed no-op (cosine scheduler overwrites), and the Reward Engineer was explicitly read-only. Agents could diagnose overtrading, DSR dominance, and direction collapse but literally could not fix anything. Additionally, training data wasn't pinned between experiments, causing baseline drift of 76% between runs.

What Was Fixed (5 Fixes)

Fix 1: Reward Weight Override Mechanism

Files: vectorized_env.py, multi_agent.py

Agents can now adjust reward component weights mid-training via set_reward_weight_override():

• Per-call bound: +/-0.05 per component
• Cumulative bound: +/-0.15 per component
• Floor: 0.01 (no component can be zeroed)
• Always re-normalized to sum=1.0
• Valid components: {direction, magnitude, pnl, stop_tp, exploration, slippage, drawdown, dsr}

Reward Engineer prompt rewritten from read-only diagnostic to actionable:

# When to adjust (ONLY when clear evidence exists):
# - Overtrading (HOLD < 20%): increase slippage +0.02 to +0.05
# - DSR dominating P&L: decrease dsr -0.02 to -0.04
# - Direction collapse (< 40% accuracy): increase direction +0.02 to +0.05
# - P&L stagnant while other metrics OK: increase pnl +0.02 to +0.05

Fix 2: Fitness Decline Gate

File: multi_agent.py

Requires 3 consecutive fitness declines before ANY intervention (entropy or weight changes). Prevents agents from intervening when training is going well.

def _is_fitness_declining(self) -> bool:
    if len(self._validation_history) < 3:
        return False
    recent = [v.get('fitness_score', 0) for v in self._validation_history[-3:]]
    return recent[-1] < recent[-2] and recent[-2] < recent[-3]

Fix 3: Tightened Entropy Bounds + Phase Gates

• Entropy bounds: [0.75x, 1.25x] (was [0.5x, 1.5x])
• Entropy phase gate: 50% (was 30%)
• Reward weight phase gate: 30% (new — earlier access since this is the main lever)
• LR adjustment: logged skip (was logged rejection with _agent_rejections increment)

Fix 4: Pinned Training Data

File: agent_validation_analysis.ipynb

DATA_START = datetime(2025, 2, 1, tzinfo=tz.UTC)
DATA_END = datetime(2026, 2, 14, tzinfo=tz.UTC)  # Fixed cutoff

Historical data is immutable — same dates = same bars = same results for same seed. Data is sliced after prefetch_all_data() (function doesn't support start/end params directly).

Fix 5: Agent Memory Improvements

File: agent_memory.py, multi_agent.py

• MultiAgentConfig.seed field — passed through to save_run_summary() (was always 0)
• Weight override history tracked per run (overrides + resulting fitness)
• _compute_patterns() aggregates weight adjustment success rate across runs
• Baseline runs saved to agent memory for proper comparison

Staged Experiment Execution

Stop after any stage that fails. Fix and re-run that stage, don't escalate.

Failed Attempts (Previous Experiments)

Key Parameters

# MultiAgentConfig (v4.2.0)
MultiAgentConfig(
    symbol=symbol,
    seed=seed,                                      # NEW: seed tracking
    enable_hyperparameter_tuner=False,               # Disabled — consolidated into Reward Engineer
    enable_reward_engineer=True,                     # NOW ACTIONABLE (was read-only)
    reward_interval=8,                               # Increased frequency
    max_cumulative_entropy_multiplier=1.25,           # Tightened from 1.5
    min_cumulative_entropy_multiplier=0.75,           # Tightened from 0.5
    no_intervention_before_pct=50.0,                  # Raised from 30.0
)

Tests Added (26 new)

• TestRewardWeightOverrideBounds (5): per-call clamp, cumulative clamp, negative clamp, invalid component, valid components
• TestRewardWeightOverrideNormalization (4): sum-to-one, floor prevents zero, weights shift
• TestAdjustRewardWeightsAction (6): action in valid set, aliases, apply action, phase gate, fitness gate, empty changes
• TestFitnessDecliningGate (6): requires 3 snapshots, declining/improving/flat, only last 3, entropy blocked
• TestLRAdjustmentNoop (2): skipped not rejected, doesn't count
• TestAgentMemoryWeightTracking (2): patterns with weight data, seed in config

References

• Plan: ~/.claude/plans/fancy-giggling-puffin.md
• Notebook: notebooks/agent_validation_analysis.ipynb (v2.0.0)
• Tests: tests/test_multi_agent.py (87 passed, 5 skipped)
• Key files: vectorized_env.py, multi_agent.py, agent_memory.py