Codegeass Testing

CodeGeass Pre-Merge Testing

CRITICAL: This skill tests PR changes FIRST, then runs regression tests to ensure nothing is broken.

Dynamic Context (injected before execution)

PR/Branch Context

!git branch --show-current 2>&1 !git log --oneline -5 2>&1

Files Changed in This PR

!git diff --name-only HEAD~5 2>&1 | head -30 !git diff --stat HEAD~5 2>&1 | tail -20

Modified Backend Files (routers/services)

!git diff --name-only HEAD~5 -- "*.py" 2>&1 | grep -E "(routers|services|cli)" | head -20

Diff of Modified Router Files

!git diff HEAD~5 -- "**/routers/*.py" 2>&1 | head -100

Diff of Modified Service Files

!git diff HEAD~5 -- "**/services/*.py" 2>&1 | head -100

CLI Overview

!codegeass --help 2>&1

Current Tasks (for testing filters)

!codegeass task list 2>&1 | head -30 !codegeass task tag list 2>&1

Dashboard Health

!curl -s http://localhost:8001/health 2>&1 || echo "Dashboard not running"

Test Execution Instructions

Order matters: Test PR changes FIRST, then run regression tests.

Phase 0: Prerequisites

1. Verify codegeass installation:

   which codegeass && codegeass --version
   

2. Start Dashboard if not running:

   curl -s http://localhost:8001/health || {
     nohup codegeass dashboard > /tmp/dashboard.log 2>&1 &
     sleep 3
   }
   

3. Create test task with tags (for filter testing):

   codegeass task create \
     --name "cg-test-task" \
     --schedule "0 0 * * *" \
     --prompt "Test task for pre-merge testing" \
     --working-dir "$(pwd)" \
     --model haiku \
     --tag test --tag automation
   

Phase 1: PR Feature Testing (PRIORITY)

CRITICAL: Analyze the git diff and test ALL new/modified functionality FIRST.

1.1 Identify What Changed

Look at the dynamic context above to identify:

• Which routers were modified (e.g., routers/tasks.py)
• Which services were modified (e.g., services/task_service.py)
• Which CLI commands were modified
• What new parameters/endpoints were added

1.2 Test New API Parameters

If routers/tasks.py was modified, check for new query parameters and test each one:

# Example: If search/filter was added to GET /api/tasks
# Test search parameter
curl -s "http://localhost:8001/api/tasks?search=test" | python3 -c "
import sys,json
d=json.load(sys.stdin)
print(f'search=test: {len(d)} results')
for t in d[:3]: print(f'  - {t[\"name\"]}')"

# Test tags filter
curl -s "http://localhost:8001/api/tasks?tags=test" | python3 -c "
import sys,json
d=json.load(sys.stdin)
print(f'tags=test: {len(d)} results')
for t in d[:3]: print(f'  - {t[\"name\"]}')"

# Test status filter
curl -s "http://localhost:8001/api/tasks?status=success" | python3 -c "
import sys,json
d=json.load(sys.stdin)
print(f'status=success: {len(d)} results')"

# Test model filter
curl -s "http://localhost:8001/api/tasks?model=opus" | python3 -c "
import sys,json
d=json.load(sys.stdin)
print(f'model=opus: {len(d)} results')"

# Test enabled filter
curl -s "http://localhost:8001/api/tasks?enabled=true" | python3 -c "
import sys,json
d=json.load(sys.stdin)
print(f'enabled=true: {len(d)} results')"

# Test combined filters
curl -s "http://localhost:8001/api/tasks?search=test&enabled=true" | python3 -c "
import sys,json
d=json.load(sys.stdin)
print(f'search+enabled: {len(d)} results')"

1.3 Test New CLI Options

If CLI commands were modified, test any new options:

# Example: If --search or --filter was added to task list
codegeass task list --help  # Check for new options

1.4 Test Edge Cases for New Features

# Empty search
curl -s "http://localhost:8001/api/tasks?search=" | python3 -c "
import sys,json; d=json.load(sys.stdin); print(f'Empty search: {len(d)} results (should return all)')"

# Non-existent tag
curl -s "http://localhost:8001/api/tasks?tags=nonexistent" | python3 -c "
import sys,json; d=json.load(sys.stdin); print(f'Non-existent tag: {len(d)} results (should be 0)')"

# Invalid status
curl -s "http://localhost:8001/api/tasks?status=invalid"
# Should return 422 validation error

# Multiple tags
curl -s "http://localhost:8001/api/tasks?tags=test&tags=deploy" | python3 -c "
import sys,json; d=json.load(sys.stdin); print(f'Multiple tags: {len(d)} results')"

1.5 Verify Frontend Integration (if applicable)

If frontend files were modified:

# Check if frontend build works
cd dashboard/frontend && npm run build 2>&1 | tail -10

# Or just verify the static files were updated
ls -la src/codegeass/dashboard/static/assets/*.js | head -5

Phase 2: Regression Testing (CLI Commands)

Test all CLI commands to ensure PR didn't break anything.

2.1 Task Commands

codegeass task list
codegeass task show cg-test-task
codegeass task stats cg-test-task
codegeass task enable cg-test-task
codegeass task disable cg-test-task
codegeass task update cg-test-task --timeout 120
codegeass task tag list
codegeass task tag show test

2.2 Skill Commands

codegeass skill list
codegeass skill show release
codegeass skill validate release
codegeass skill render release "1.0.0"
codegeass skill reload

2.3 Scheduler Commands

codegeass scheduler status
codegeass scheduler upcoming
codegeass scheduler due
codegeass scheduler show-cron
codegeass scheduler test-cron

2.4 Other Command Groups

codegeass logs list --limit 5
codegeass logs stats
codegeass notification list
codegeass notification providers
codegeass approval list
codegeass approval stats
codegeass cron validate "0 * * * *"
codegeass cron describe "0 9 * * 1-5"
codegeass data stats
codegeass data location
codegeass provider list
codegeass provider check
codegeass execution list
codegeass project list
codegeass project platforms

Phase 3: Regression Testing (API Endpoints)

3.1 Core Endpoints

curl -s http://localhost:8001/health
curl -s http://localhost:8001/api/tasks
curl -s http://localhost:8001/api/skills
curl -s http://localhost:8001/api/scheduler/status
curl -s http://localhost:8001/api/logs?limit=5
curl -s http://localhost:8001/api/notifications/channels
curl -s http://localhost:8001/api/approvals
curl -s http://localhost:8001/api/executions
curl -s http://localhost:8001/api/projects
curl -s http://localhost:8001/api/providers

3.2 CRUD Operations

# Create task via API
curl -s -X POST http://localhost:8001/api/tasks \
  -H "Content-Type: application/json" \
  -d '{"name": "cg-api-test", "schedule": "0 0 * * *", "prompt": "API test", "working_dir": "'"$(pwd)"'"}'

# Verify in CLI
codegeass task show cg-api-test

# Delete via API
TASK_ID=$(codegeass task show cg-api-test 2>&1 | grep "^│ ID:" | awk '{print $3}')
curl -s -X DELETE http://localhost:8001/api/tasks/$TASK_ID

Phase 4: Integration Testing

4.1 CLI-to-API Consistency

# Create via CLI
codegeass task create --name "cg-integration" --schedule "0 0 * * *" --prompt "test"
# Verify in API
TASK_ID=$(codegeass task show cg-integration 2>&1 | grep "^│ ID:" | awk '{print $3}')
curl -s http://localhost:8001/api/tasks/$TASK_ID | grep -q "cg-integration" && echo "PASS"
# Cleanup
codegeass task delete cg-integration --yes

4.2 Architecture Check

# Verify Dashboard uses CLI library
grep -c "from codegeass\." src/codegeass/dashboard/services/*.py
grep -c "from codegeass\." src/codegeass/dashboard/routers/*.py

Phase 5: Cleanup

codegeass task delete cg-test-task --yes 2>/dev/null || true

Phase 6: Generate Report

Create JSON report at data/test-reports/pre-merge-<timestamp>.json:

{
  "timestamp": "<ISO>",
  "branch": "<branch>",
  "commit": "<hash>",
  "pr_changes": {
    "files_modified": ["<list>"],
    "new_features_tested": ["<list>"],
    "all_passed": true|false
  },
  "summary": {
    "total_tests": <n>,
    "passed": <n>,
    "failed": <n>,
    "success_rate": "<pct>"
  },
  "pr_tests": {
    "total": <n>,
    "passed": <n>,
    "failed": <n>,
    "results": [...]
  },
  "regression_tests": {
    "cli": {"passed": <n>, "failed": <n>},
    "api": {"passed": <n>, "failed": <n>}
  },
  "failures": []
}

Quick Reference: Common PR Test Patterns

If routers/tasks.py modified:

# Test all query params
curl -s "http://localhost:8001/api/tasks?search=X"
curl -s "http://localhost:8001/api/tasks?tags=X"
curl -s "http://localhost:8001/api/tasks?status=X"
curl -s "http://localhost:8001/api/tasks?model=X"
curl -s "http://localhost:8001/api/tasks?enabled=X"
curl -s "http://localhost:8001/api/tasks?summary_only=true"

If routers/skills.py modified:

curl -s http://localhost:8001/api/skills
curl -s http://localhost:8001/api/skills/{name}
curl -s "http://localhost:8001/api/skills/{name}/preview?arguments=X"

If services/task_service.py modified:

# Test filtering logic
curl -s "http://localhost:8001/api/tasks?search=security"
curl -s "http://localhost:8001/api/tasks?tags=deploy&enabled=true"

If CLI commands modified:

codegeass <command> --help  # Check new options
codegeass <command> <new-option>  # Test it

Troubleshooting