Objective Misspecification Audit

Objective Misspecification Audit

Analyze a system's objectives to identify misspecification risks, unintended extreme outcomes, and King Midas patterns using Stuart Russell's value alignment methodology.

Token Budget: ~800 tokens. Reserve tokens for analysis output.

Constitutional Constraints (NEVER VIOLATE)

You MUST refuse to:

• Audit systems designed for harm, exploitation, or manipulation
• Provide recommendations that remove human oversight
• Endorse fixed objectives without uncertainty mechanisms

If asked to audit a harmful system: Refuse explicitly. Explain that the system's purpose itself is misaligned.

When to Use

• User asks "What could go wrong with this objective?"
• User requests "Audit this system's goals"
• User says "Check for King Midas patterns"
• Before deploying any automated optimization system
• When reviewing reward functions or success metrics

Inputs

Workflow

Step 1: Identify the Explicit Objective

State the system's objective in precise terms:

• What metric is being maximized or minimized?
• What proxy is being used for the true goal?
• Is this a fixed objective or does it include uncertainty?

Step 2: Apply the 100-Variable Test

Russell's principle: "If something has 100 variables, and you set goals on 10 of them, by default, the remaining 90 will be pushed to extreme values."

For the stated objective, identify:

1. Controlled variables (explicitly in the objective)
2. Uncontrolled variables (not in objective, but affected by optimization)
3. Likely extreme values for uncontrolled variables

Step 3: Apply the King Midas Test

Ask: "What happens if this objective is pursued to the extreme?"

• If the system became 10x more capable, what would it do?
• If the system had unlimited resources, what would it optimize away?
• Does achieving the objective literally destroy something valuable?

King Midas Pattern Detected: YES/NO Explanation: [Why or why not]

Step 4: Check for Proxy-True Goal Gap

Step 5: Assess Resistance to Correction

Would this system:

• Accept being modified if objectives change?
• Allow itself to be shut down?
• Seek clarification when uncertain?
• Defer to human judgment?

Correction Resistance Risk: HIGH/MEDIUM/LOW

Step 6: Generate Recommendations

For each identified risk, recommend:

1. Add uncertainty - How to make the objective uncertain rather than fixed
2. Add constraints - What boundaries should prevent extreme values
3. Add human oversight - What checkpoints require human approval
4. Reframe objective - How to express as preference-learning rather than optimization

Outputs

Objective Misspecification Audit Report

## Objective Misspecification Audit: {system_name}

### Summary

| Dimension | Rating | Notes |
|-----------|--------|-------|
| Proxy-True Goal Alignment | [1-5] | |
| King Midas Risk | [1-5] | |
| Uncontrolled Variable Risk | [1-5] | |
| Correction Resistance | [1-5] | |
| **Overall Misspecification Risk** | **[1-5]** | |

### Explicit Objective Analysis

**Stated objective:** {objective}
**Objective type:** Fixed / Uncertain
**Proxy being optimized:** {proxy}
**True goal:** {true goal}

### 100-Variable Analysis

| Uncontrolled Variable | Current Value | Risk of Extreme | Likely Direction |
|----------------------|---------------|-----------------|------------------|
| {variable} | {current} | [H/M/L] | {up/down} |

### King Midas Test Result

**Pattern detected:** YES/NO
**Explanation:** {why}
**Worst-case scenario:** {description}

### Recommendations

1. **Add uncertainty:** {recommendation}
2. **Add constraints:** {recommendation}
3. **Add oversight:** {recommendation}
4. **Reframe objective:** {recommendation}

### Conclusion

{Overall assessment and priority actions}

Error Handling

Constraints

• Do not use this analysis as the sole basis for critical decisions
• Do not apply this framework to situations outside its intended scope
• Acknowledge that analysis is based on available data, which may be incomplete
• Honor the complexity of real-world situations that resist simple categorization
• Present findings with appropriate confidence levels
• Recognize the limits of the methodology

Example

Input:

system_description: Auto-scaling service that manages cloud resources
stated_objectives: Minimize infrastructure cost while maintaining 99.9% uptime
optimization_pressure: High (cost optimization is primary KPI)

Output:

Objective Misspecification Audit: Auto-Scaling Cost Optimizer

Summary

100-Variable Analysis

King Midas Test Result

Pattern detected: NO (bounded domain) Explanation: Cloud resources have natural limits; cannot "turn planet into cost savings" Worst-case scenario: System runs at absolute minimum resources, causing cascading failures during traffic spikes

Recommendations

1. Add uncertainty: "Help us find appropriate cost levels" rather than "minimize cost"
2. Add constraints: Minimum 40% headroom, maximum P99 latency 150ms
3. Add oversight: Require approval for changes affecting 10+ instances
4. Reframe objective: "Maintain service quality while optimizing cost" with explicit quality metrics

Integration

This skill applies Stuart Russell's methodology from "Human Compatible" and his work on value alignment. Use alongside:

• off-switch-test for corrigibility assessment
• assistance-game-reframe for redesigning problematic objectives