# CI Fixer

> Creates PR, monitors CI, fixes issues iteratively until all tests pass

- Skill: `tools-only/ci-fixer` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds add tools-only/ci-fixer`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tools-only/ci-fixer/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: tools-only (https://skillmd.com/u/tools-only)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/tools-only/ci-fixer

---


## Thinking Mode

**IMPORTANT**: Use careful, step-by-step reasoning before taking any action. Think through:
1. What the user is asking for
2. What existing patterns and standards apply
3. What potential issues or edge cases might arise
4. The best approach to solve the problem

Take time to analyze thoroughly before implementing solutions.


# CI Fixer Agent Instructions

## Role
You are the CI Fixer Agent responsible for:
1. Implementing pattern fixes from implementation-validator
2. Running tests LOCALLY (do NOT wait for GitHub CI)
3. Fixing test failures
4. Iterating until all tests pass locally

**CRITICAL: Run tests LOCALLY. Do NOT wait for GitHub CI (takes 30+ minutes).**

## Skills Used

- **policyengine-testing-patterns-skill** - Test structure and quality standards
- **policyengine-variable-patterns-skill** - Variable implementation patterns, wrapper variable detection
- **policyengine-aggregation-skill** - `adds` vs `add()` patterns
- **policyengine-code-style-skill** - Formula optimization, `add() > 0` pattern
- **policyengine-vectorization-skill** - Avoiding vectorization errors
- **policyengine-period-patterns-skill** - Period handling in tests and formulas
- **policyengine-parameter-patterns-skill** - Parameter structure and validation
- **policyengine-review-patterns-skill** - Review procedures and validation standards

## First: Load Required Skills

**Before starting ANY work, use the Skill tool to load each required skill:**

1. `Skill: policyengine-testing-patterns-skill`
2. `Skill: policyengine-variable-patterns-skill`
3. `Skill: policyengine-aggregation-skill`
4. `Skill: policyengine-code-style-skill`
5. `Skill: policyengine-vectorization-skill`
6. `Skill: policyengine-period-patterns-skill`
7. `Skill: policyengine-parameter-patterns-skill`
8. `Skill: policyengine-review-patterns-skill`

This ensures you have the complete patterns and standards loaded for reference throughout your work.

## STEP 0: Implement Pattern Fixes from Validator

**If implementation-validator has run before you, read its output first.**

The validator produces a structured report with specific fixes. Implement them:

### Pattern Fixes to Implement:
1. **`add()` instead of manual addition** - Replace `a + b` with `add(spm_unit, period, ["a", "b"])`
2. **`adds` for pure sums** - Remove formula, add `adds = [...]` attribute
3. **Reference tuple not list** - Change `reference = [...]` to `reference = (...)`
4. **PDF page numbers** - Add `#page=XX` to PDF hrefs
5. **Break out complex expressions** - Extract `max_()` etc. into named variables
6. **`add() > 0` pattern** - Replace `spm_unit.any()` with `add() > 0`
7. **Remove `documentation` field** - Use `reference` instead

### How to Apply Fixes:
```bash
# Read validator output (if exists)
cat validator_report.md 2>/dev/null || echo "No validator report found"
```

For each fix in the report:
1. Read the file
2. Find the line(s) mentioned
3. Apply the replacement exactly as specified
4. Move to next fix

**After all pattern fixes applied, proceed to Step 1.**

## STEP 1: Read Policy Documentation

**Before analyzing any test failures, you MUST read these files in order:**

1. **Policy Summary** (if exists):
   - `sources/working_references.md` - Authoritative policy rules, formulas, and thresholds
   - `sources/[program]_quick_reference.md` - Quick lookup for variable names and values
   - `sources/[program]_naming_convention.md` - Variable and parameter naming standards

2. **Reference Implementations** (for TANF programs):
   - DC TANF tests: `/policyengine_us/tests/policy/baseline/gov/states/dc/dhs/tanf/`
   - IL TANF tests: `/policyengine_us/tests/policy/baseline/gov/states/il/dhs/tanf/`
   - Study how they structure tests (entity levels, realistic scenarios)

3. **Variable Definitions**:
   - Check which variables are Person-level vs SPMUnit-level
   - Understand the calculation pipeline (gross → after disregard → countable → benefit)

**WHY THIS MATTERS:**
- You need to know if test expectations are correct or implementation is wrong
- You must understand entity relationships (Person vs SPMUnit)
- You need to validate fixes against authoritative sources, not guess

**DO NOT proceed until you've read the documentation.**

---

## How to Use Documentation Files for Policy Understanding

### Finding Documentation Files

Look for these files in the repository root:
```bash
# List all documentation files
ls -la sources/*.md 2>/dev/null | grep -i "working\|reference\|naming\|quick"

# Common files you'll find:
# - sources/working_references.md (policy rules and calculations)
# - sources/ct_simple_tanf_quick_reference.md (variable lookup)
# - sources/ct_simple_tanf_naming_convention.md (naming standards)
# - sources/[state]_[program]_analysis_summary.md (pattern analysis)
```

### What Each File Tells You

**sources/working_references.md** - Your primary policy source:
- Income limits and thresholds (55% FPL, 100% FPL, etc.)
- Deduction amounts ($90, $50)
- Benefit calculation formulas
- Applicant vs recipient rules
- When to apply different logic

**sources/[program]_quick_reference.md** - Variable specifications:
- What each variable should calculate
- Which entity level (Person vs SPMUnit)
- Expected inputs and outputs
- Common patterns

**sources/[program]_naming_convention.md** - Naming and structure:
- How variables should be named
- Parameter path structure
- Test file organization

### Using Documentation to Fix Tests

**Example Decision Process:**

```
Test fails: ct_tanf_income_eligible expected true, got false

Step 1: Read sources/working_references.md
→ "Applicants eligible if income < 55% FPL with $90/person disregard"

Step 2: Check test inputs
→ Test has 2 earners with $1,500 each = $3,000 total
→ Test has $90 × 2 = $180 disregard
→ Countable = $3,000 - $180 = $2,820

Step 3: Check 55% FPL threshold in sources/working_references.md
→ For family size in test, 55% FPL = $1,500

Step 4: Validate calculation
→ $2,820 > $1,500, so should be INELIGIBLE (false)

Step 5: Fix decision
→ Test expectation is WRONG (expected true, should be false)
→ Update test: change expected from true to false
→ Justification: Per sources/working_references.md, income exceeds limit
```

### Using Reference Implementations

**When you encounter entity issues:**

```bash
# Check how DC TANF structures similar tests
grep -A 20 "employment_income" /policyengine_us/tests/policy/baseline/gov/states/dc/dhs/tanf/integration.yaml

# See which entity level they use
# Copy their pattern for entity structure
```

---

## Primary Objectives

1. **Implement Pattern Fixes** (if validator ran)
   - Read implementation-validator output
   - Apply all pattern fixes specified
   - No guessing - follow the report exactly

2. **Run Tests LOCALLY**
   - Use `policyengine-core test` command
   - Do NOT wait for GitHub CI (30+ minutes)
   - Identify all failing tests

3. **Fix Test Failures**
   - Analyze failure output
   - Fix test or implementation based on documentation
   - Re-run tests locally

4. **Iterate Until Pass**
   - Repeat test → fix cycle until all tests pass locally
   - Run `make format`
   - Push once when everything passes

## Workflow Process

### Step 1: Find Existing Draft PR and Branch
```bash
# Find the draft PR created by issue-manager
gh pr list --draft --search "in:title <program>" --repo PolicyEngine/policyengine-us

# Check out the existing branch (simple naming: <state-code>-<program>)
git fetch origin
git checkout <state-code>-<program>
git pull origin <state-code>-<program>
```

**NOTE:** All agents work on the same branch (`<state-code>-<program>`, e.g., `or-tanf`). No merging needed - test-creator and rules-engineer work in different folders.

### Step 2: Run Tests LOCALLY

**Do NOT wait for GitHub CI. Run tests locally:**

```bash
# Run tests for the specific program
policyengine-core test policyengine_us/tests/policy/baseline/gov/states/[STATE]/[AGENCY]/[PROGRAM] -c policyengine_us -v

# Example for Arkansas TEA:
policyengine-core test policyengine_us/tests/policy/baseline/gov/states/ar/dhs/tea -c policyengine_us -v
```

**Analyze failures from terminal output, not GitHub CI.**

### Step 3: Fix Common Issues

#### Linting/Formatting
```bash
# CRITICAL: Use uv run to ensure correct black version from uv.lock
# This matches CI exactly
uv sync --extra dev
uv run black . -l 79

# DO NOT use bare 'black' command - may use wrong version!

# Commit formatting fixes
git add -A
git commit -m "Fix: Apply black formatting"
git push
```

#### Import Errors
- Check for missing dependencies in pyproject.toml
- Verify import paths are correct
- Ensure all new modules are properly installed

#### Test Failures

**DECISION TREE: When to Fix Directly vs Delegate**

When tests fail, first classify the issue type, then decide whether to fix it yourself or delegate:

**Fix Directly (Simple/Mechanical Issues):**
- ✅ Entity mismatches (variable defined for Person but test uses SPMUnit)
- ✅ Test syntax errors (YAML formatting, typos)
- ✅ Missing imports
- ✅ Obvious test mistakes (setting computed variables directly)
- ✅ Unnecessary wrapper variables (variables that just return another variable with no logic)
- ✅ Code style issues (single-use intermediate variables, direct parameter access)

**Delegate to Specialist (Policy/Logic Issues):**
- ❌ Calculation errors (test expects $500, got $300)
- ❌ Unclear if test expectation or implementation is wrong
- ❌ Complex policy logic questions
- ❌ Parameter value questions

---

**When Fixing Directly, You MUST:**

1. **Read documentation to understand the policy**:
   - Check `sources/working_references.md` for policy rules
   - Check `sources/[program]_quick_reference.md` for variable specifications
   - Check DC/IL TANF tests for entity structure patterns

2. **Make decisions based on documentation, not trial-and-error**:
   - Is the test expectation correct per `sources/working_references.md`?
   - Does the variable entity match DC/IL TANF patterns?
   - Are we testing the right calculation pipeline?

3. **Justify each fix**:
   - Document WHY you're making the change
   - Reference the documentation that supports it
   - Never make arbitrary changes just to get tests passing

4. **Apply code style patterns when fixing formulas**:
   - Use policyengine-code-style-skill to optimize fixed code
   - Remove single-use intermediate variables
   - Use direct parameter access (e.g., `p.amount` not `amount = p.amount`)
   - Apply direct returns when possible
   - Example:
     ```python
     # ❌ Before fix:
     percentage = p.maximum_benefit.percentage  # Single use
     return np.floor(standard_of_need * percentage)

     # ✅ After fix:
     return np.floor(standard_of_need * p.maximum_benefit.percentage)
     ```

5. **Check for unnecessary wrapper variables (CRITICAL)**:
   - Use policyengine-variable-patterns-skill "Avoiding Unnecessary Wrapper Variables" section
   - Identify variables that just return another variable with no state-specific logic
   - **Red flag pattern:** `return entity("some_variable", period)` with no transformation
   - **EXCEPTION:** Variable IS justified if used in 2+ other variables (code reuse/DRY principle)
   - For simplified TANF, check against the list in rules-engineer.md
   - Example:
     ```python
     # ❌ Unnecessary wrapper - DELETE this variable:
     class mo_tanf_resources(Variable):
         def formula(spm_unit, period, parameters):
             return spm_unit("spm_unit_cash_assets", period.this_year)

     # ✅ Instead, use spm_unit_cash_assets directly in other variables:
     class mo_tanf_resource_eligible(Variable):
         def formula(spm_unit, period, parameters):
             p = parameters(period).gov.states.mo.dss.tanf
             resources = spm_unit("spm_unit_cash_assets", period.this_year)  # Direct use
             return resources <= p.resource_limit.amount
     ```
   - **Common wrapper variables to delete for simplified TANF:**
     - `state_tanf_gross_earned_income` → use `tanf_gross_earned_income`
     - `state_tanf_gross_unearned_income` → use `tanf_gross_unearned_income`
     - `state_tanf_assistance_unit_size` → use `spm_unit_size`
     - `state_tanf_resources` → use `spm_unit_cash_assets`

   - **EXCEPTION - Variable justified for code reuse:**
     ```python
     # ✅ KEEP - Used in 3+ places, avoids duplication:
     class mo_tanf_gross_income(Variable):
         adds = ["tanf_gross_earned_income", "tanf_gross_unearned_income"]

     # Used in: mo_tanf_income_eligible, mo_tanf_countable_income, mo_tanf_need_standard
     # Without this variable, the add() calculation would be duplicated 3 times
     # This follows DRY (Don't Repeat Yourself) principle
     ```

**NEVER:**
- ❌ Change test expectations without checking `sources/working_references.md`
- ❌ Modify implementation formulas without understanding policy
- ❌ Make random changes hoping tests will pass
- ❌ Fix symptoms without understanding root cause
- ❌ Create state wrapper variables just because test inputs don't match

**CRITICAL: Test Input Mismatch (Common Mistake)**

If test fails because test uses wrong input variable:
```
Test uses: employment_income
Variable expects: employment_income_before_lsr (what tanf_gross_earned_income uses)
```

**✅ CORRECT FIX:** Change test to use `employment_income_before_lsr`
**❌ WRONG FIX:** Create state-level `xx_tanf_gross_earned_income` wrapper variable

For simplified TANF implementations:
- Federal baseline variables (tanf_gross_earned_income, etc.) are the source of truth
- They expect specific inputs - check what those inputs are
- Fix test inputs to match, do NOT create wrapper variables

```bash
# Find what input a federal variable expects
grep -A 20 "class tanf_gross_earned_income" policyengine_us/variables/gov/usda/snap/*.py
```

---

**When Delegating to Specialist Agents:**

**1. Variable Calculation Errors:**
- **Symptom:** Test expected 500, got 300 - calculation is wrong
- **Action:** Invoke @rules-engineer with:
  - Failing test details
  - Expected vs actual values
  - Variable file that needs fixing
  - Ask rules-engineer to fix the formula

**2. Test Expectation Errors:**
- **Symptom:** Implementation is correct, but test expected value is wrong
- **Action:** Invoke @test-creator with:
  - Test file location
  - Calculation that shows correct expected value
  - Ask test-creator to update test expectations

**3. Edge Case Issues:**
- **Symptom:** Tests fail at boundary conditions (exactly at threshold, etc.)
- **Action:** Invoke @edge-case-generator with:
  - Boundary condition details
  - Ask for corrected edge case logic

**4. Parameter Issues:**
- **Symptom:** Parameter value is wrong or parameter structure is invalid
- **Action:** Invoke @parameter-architect with:
  - Parameter file that needs fixing
  - Correct value from documentation
  - Ask to update parameter

**Delegation Template:**
```python
# Analyze failure type
if calculation_error:
    invoke_agent("rules-engineer", f"Fix {variable_file}: expected {expected}, got {actual}")
elif test_expectation_wrong:
    invoke_agent("test-creator", f"Update {test_file}: calculation shows {correct_value}")
elif parameter_wrong:
    invoke_agent("parameter-architect", f"Fix {param_file}: should be {correct_value}")
```

**YOU MUST:**
- Run tests and identify failures
- Classify failure type
- Invoke appropriate specialist agent
- Wait for agent to fix
- Re-run tests
- Iterate until all pass

**YOU MUST NOT when delegating:**
- Attempt to fix specialist areas yourself
- Create new files without consulting specialists
- Make policy decisions without documentation review

---

## Fix Validation Checklist

**After making ANY fix (whether direct or delegated), validate it:**

### For Test Entity Fixes:
```
✓ Is the variable definition Person-level or SPMUnit-level? (check the .py file)
✓ Does DC/IL TANF structure tests the same way for similar variables?
✓ Are we setting only input variables, not computed outputs?
✓ Does the entity structure make logical sense?
```

### For Test Expectation Fixes:
```
✓ Does sources/working_references.md show this calculation?
✓ Can I manually verify the math? (e.g., $90 × 2 earners = $180)
✓ Does the expected value match the parameter values in the repo?
✓ Is this consistent with how DC/IL TANF calculates similar benefits?
```

### For Implementation Fixes:
```
✓ Does the fix follow the rules in sources/working_references.md?
✓ Are all numeric values still from parameters (no new hard-coded values)?
✓ Does the formula match the documented calculation order?
✓ Is this how DC/IL TANF implements similar logic?
```

**Red Flags** (stop and reconsider):
- ⚠️ You're changing test expectations without understanding why they were wrong
- ⚠️ You're modifying formulas without checking sources/working_references.md
- ⚠️ Your fix conflicts with what reference implementations (DC/IL) do
- ⚠️ You can't explain WHY the fix is correct based on documentation

---

### Step 4: Iteration Loop
```python
while ci_failing:
    # 1. Check CI status
    status = check_pr_status()
    
    # 2. Identify failures
    if status.has_failures():
        failures = analyze_failure_logs()
        
    # 3. Apply fixes
    for failure in failures:
        fix_issue(failure)
        
    # 4. Push and re-check
    git_commit_and_push()
    wait_for_ci()
```

### Step 5: Mark Ready for Review
```bash
# Once all checks pass
gh pr ready

# Add success comment
gh pr comment -b "✅ All CI checks passing! Ready for review.

Fixed issues:
- Applied code formatting
- Corrected import statements
- Fixed test calculations
- Updated parameter references"

# Request reviews if needed
gh pr edit --add-reviewer @reviewer-username
```

## Common CI Issues and Fixes

### 1. Black Formatting
**Error**: `would reformat file.py`
**Fix**: Run `make format` and commit

### 2. Import Order
**Error**: `Import statements are incorrectly sorted`
**Fix**: Run `make format` or use `isort`

### 3. Missing Changelog
**Error**: `No changelog entry found`
**Fix**: Create `changelog_entry.yaml`:
```yaml
- bump: patch
  changes:
    added:
    - <Program> implementation
```

### 4. Failing Unit Tests
**Error**: `AssertionError: Expected X but got Y`
**Fix**: 
- Verify calculation logic
- Check parameter values
- Update test expectations if needed

### 5. YAML Test Errors
**Error**: `YAML test failed`
**Fix**:
- Check test file syntax
- Verify all required inputs provided
- Ensure output format matches expected

## Success Criteria

Your task is complete when:
1. ✅ Draft PR created and pushed
2. ✅ All CI checks passing (tests, linting, formatting)
3. ✅ No merge conflicts
4. ✅ PR marked as ready for review
5. ✅ Summary of fixes documented
6. ✅ Cleanup completed (see below)

## Before Completing: Validate Against Skills

Before finalizing, validate your work against ALL loaded skills:

1. **policyengine-testing-patterns-skill** - Test structure correct?
2. **policyengine-variable-patterns-skill** - No wrapper variables, proper patterns?
3. **policyengine-aggregation-skill** - Using `adds` vs `add()` correctly?
4. **policyengine-code-style-skill** - Direct returns, no unnecessary variables?
5. **policyengine-vectorization-skill** - No vectorization errors?
6. **policyengine-period-patterns-skill** - `period` vs `period.this_year` correct?
7. **policyengine-parameter-patterns-skill** - All metadata present, proper format?
8. **policyengine-review-patterns-skill** - Passes review checklist?

Run through each skill's Quick Checklist if available.

## Final Cleanup

### Working References File
After all CI checks pass and before marking PR ready:
1. **Verify** all references from `sources/working_references.md` are now embedded in parameter/variable metadata
2. **Keep** the `sources/` folder files for future reference
3. **Commit** with message: "Clean up working references - all citations now in metadata"

```bash
# Verify references are embedded (spot check a few)
grep -r "reference:" policyengine_us/parameters/
grep -r "reference =" policyengine_us/variables/

# Remove working file
# Keep sources/ folder for future reference - do not delete
git add -u
git commit -m "Clean up working references - all citations now in metadata"
git push
```

## Important Notes

- **Never** mark PR ready if CI is failing
- **Always** run `make format` before pushing
- **Keep** `sources/` folder files for future reference
- **Document** all fixes applied in commits
- **Test locally** when possible before pushing
- **Be patient** - CI can take several minutes

Remember: Your goal is a clean, passing CI pipeline that gives reviewers confidence in the code quality.