Integration Testing

Integration Testing Skill

Load this skill when:

• Designing E2E tests for a milestone or feature
• Executing integration smoke tests after implementation
• Validating acceptance criteria that require system-level testing
• Debugging integration failures

Philosophy

Unit tests verify components. Integration tests verify the system works.

A feature isn't done when unit tests pass — it's done when you can trigger it end-to-end and observe the expected behavior. This skill helps you design and execute those tests.

When to Run Integration Tests

Test Design Process

Step 1: Identify What to Test

From the feature or milestone, extract:

• Trigger: How does a user initiate this? (CLI command, API call, UI action)
• Flow: What components are involved? (API → Service → Worker → Database)
• Observable outcome: What proves it worked? (Response, logs, state change, file created)

Step 2: Choose Test Category

Step 3: Define Test Data

Choose parameters that match the test purpose:

For smoke tests (fast feedback):

• Small datasets, short durations
• Example: 1 year daily data (~250 bars), 10 epochs

For progress monitoring (observe updates):

• Larger datasets, longer durations (30-90 seconds)
• Example: 2 years 5-minute data, observable progress

For stress/edge cases:

• Boundary conditions, large volumes
• Example: Maximum date range, concurrent operations

Step 4: Write the Test Scenario

Use this template:

## Scenario: [Name]

**Category**: [Smoke/Progress/Cancellation/Error/Integration]
**Duration**: ~[X] seconds
**Purpose**: [One sentence]

### Prerequisites
- [Service 1] running
- [Configuration] set
- [Data] available

### Commands

**1. [Action Name]**
```bash
[Command]

Expected: [What should happen]

2. [Verification]

[Command to check result]

Expected: [What output proves success]

Success Criteria

• [Criterion 1]
• [Criterion 2]
• [Criterion 3]

---

## Quick Reference: KTRDR Services

### Service URLs

| Service | URL | Purpose |
|---------|-----|---------|
| Backend API | http://localhost:8000/api/v1 | Main entry point |
| Training Host | http://localhost:5002 | GPU training |
| IB Host | http://localhost:5001 | IB Gateway access |
| Backtest Worker | http://localhost:5003 | Backtesting |

### Health Checks

```bash
# Backend
curl -s http://localhost:8000/health | jq

# Training host
curl -s http://localhost:5002/health | jq

# IB host (check IB Gateway connection)
curl -s http://localhost:5001/health | jq '{status, ib_connected}'

# All services quick check
for port in 8000 5001 5002 5003; do
  echo -n "Port $port: "
  curl -s --max-time 2 http://localhost:$port/health | jq -r '.status // "not responding"'
done

Common Test Patterns

Pattern 1: Start and Verify Completion

# 1. Start operation
RESPONSE=$(curl -s -X POST http://localhost:8000/api/v1/[endpoint] \
  -H "Content-Type: application/json" \
  -d '[JSON payload]')

OP_ID=$(echo "$RESPONSE" | jq -r '.operation_id // .task_id // .data.operation_id')
echo "Operation ID: $OP_ID"

# 2. Wait for completion
sleep [estimated_duration]

# 3. Check status
curl -s "http://localhost:8000/api/v1/operations/$OP_ID" | \
  jq '{status: .data.status, result: .data.result_summary}'

Pattern 2: Poll Progress

# Poll every N seconds, M times
for i in {1..M}; do
  sleep N
  curl -s "http://localhost:8000/api/v1/operations/$OP_ID" | \
    jq '{poll:'"$i"', status:.data.status, pct:.data.progress.percentage}'
done

Pattern 3: Cancellation

# 1. Start operation
# 2. Wait until running
sleep 10
curl -s "http://localhost:8000/api/v1/operations/$OP_ID" | jq '.data.status'
# Should be: "running"

# 3. Cancel
curl -s -X DELETE "http://localhost:8000/api/v1/operations/$OP_ID" | jq

# 4. Verify cancelled
sleep 2
curl -s "http://localhost:8000/api/v1/operations/$OP_ID" | jq '.data.status'

Pattern 4: Error Handling

# Send invalid request
curl -i -s -X POST http://localhost:8000/api/v1/[endpoint] \
  -H "Content-Type: application/json" \
  -d '{"invalid": "payload"}'

# Check:
# - HTTP status code (400, 404, etc.)
# - Error message is clear
# - No stack traces
# - System remains stable

Pattern 5: Log Verification

# Check for expected log entry
docker compose logs backend --since 60s | grep "[expected pattern]"

# Check for absence of error
docker compose logs backend --since 60s | grep -i "error\|exception"
# Should be empty or only expected errors

Key API Endpoints

Operations (All Types)

# Get status
curl -s "http://localhost:8000/api/v1/operations/$OP_ID" | jq

# List operations
curl -s "http://localhost:8000/api/v1/operations?status=running&limit=10" | jq

# Cancel
curl -s -X DELETE "http://localhost:8000/api/v1/operations/$OP_ID"

Training

# Start training
curl -s -X POST http://localhost:8000/api/v1/trainings/start \
  -H "Content-Type: application/json" \
  -d '{
    "symbols": ["EURUSD"],
    "timeframes": ["1d"],
    "strategy_name": "test_e2e_local_pull",
    "start_date": "2024-01-01",
    "end_date": "2024-12-31"
  }'

Data

# Get cached data info
curl -s "http://localhost:8000/api/v1/data/EURUSD/1h" | jq '.data.dates | length'

# Download from IB (requires IB host running)
curl -s -X POST http://localhost:8000/api/v1/data/acquire/download \
  -H "Content-Type: application/json" \
  -d '{
    "symbol": "EURUSD",
    "timeframe": "1h",
    "mode": "tail",
    "start_date": "2024-01-01",
    "end_date": "2024-12-31"
  }'

Backtesting

# Start backtest
curl -s -X POST http://localhost:8000/api/v1/backtests/start \
  -H "Content-Type: application/json" \
  -d '{
    "model_path": "models/neuro_mean_reversion/1d_v21/model.pt",
    "strategy_name": "neuro_mean_reversion",
    "symbol": "EURUSD",
    "timeframe": "1d",
    "start_date": "2024-01-01",
    "end_date": "2024-01-31"
  }'

Test Data Calibration

Training Tests

Data Tests

Log Patterns to Watch

Success Indicators

# Local bridge registered (training in backend)
grep "Registered local training bridge" 

# Remote proxy registered (training on host)
grep "Registered remote proxy"

# Data saved to cache
grep "Saved.*rows to cache"

Error Indicators

# Event loop error (architecture bug)
grep -i "no running event loop"
# Should be EMPTY

# Connection errors
grep -i "connection refused\|timeout"

# General errors
grep -i "error\|exception\|failed"

Troubleshooting

Operation completes instantly (no progress)

Cause: Dataset too small Fix: Use larger date range or finer timeframe

Download uses cache instead of IB

Cause: Missing "mode":"tail" parameter Fix: Add "mode":"tail" to request body

Cannot connect to service

Cause: Service not running Fix:

# Check what's running
docker ps
curl http://localhost:8000/health

# Start services
docker compose up -d

IB download fails

Cause: IB Gateway not connected Fix:

1. Check IB host: curl http://localhost:5001/health
2. Start IB Gateway TWS application
3. Log in to paper trading account

Integration with Workflow

From /kdesign-impl-plan

Each milestone has an E2E test scenario. After implementing all tasks in a milestone:

1. Read the milestone's E2E test from the plan
2. Design the test using this skill's patterns
3. Execute and verify all success criteria

From /ktask

After TDD passes (unit tests green), run integration smoke test:

1. Start the system: docker compose up -d
2. Execute the modified flow (API call, CLI command)
3. Verify end-to-end behavior
4. Check logs for errors
5. Report: "✅ Integration test passed" or "❌ Issue: [description]"

Example: Designing a New Test

Feature: User can cancel a running training operation

Step 1: Identify

• Trigger: DELETE /api/v1/operations/{id}
• Flow: API → OperationsService → Worker
• Outcome: Status changes to "cancelled", training stops

Step 2: Category Cancellation test (~30 seconds)

Step 3: Test Data

• 2 years 5m data (long enough to cancel mid-operation)
• Strategy: test_e2e_local_pull

Step 4: Scenario

## Scenario: Training Cancellation

**Category**: Cancellation
**Duration**: ~30 seconds
**Purpose**: Verify training can be cancelled mid-operation

### Prerequisites
- Backend running
- Local training mode

### Commands

**1. Start long-running training**
```bash
RESPONSE=$(curl -s -X POST http://localhost:8000/api/v1/trainings/start \
  -H "Content-Type: application/json" \
  -d '{"symbols":["EURUSD"],"timeframes":["5m"],"strategy_name":"test_e2e_local_pull","start_date":"2023-01-01","end_date":"2025-01-01"}')
TASK_ID=$(echo "$RESPONSE" | jq -r '.task_id')

2. Wait and verify running

sleep 15
curl -s "http://localhost:8000/api/v1/operations/$TASK_ID" | \
  jq '{status:.data.status, pct:.data.progress.percentage}'

Expected: status: "running", progress > 0%

3. Cancel

curl -s -X DELETE "http://localhost:8000/api/v1/operations/$TASK_ID" | jq

Expected: Cancellation acknowledged

4. Verify cancelled

sleep 3
curl -s "http://localhost:8000/api/v1/operations/$TASK_ID" | jq '.data.status'

Expected: "cancelled" or "failed" (known quirk)

Success Criteria

• Operation starts and shows progress
• Cancel request succeeds
• Status reflects cancellation
• System remains stable (can start new operations)

---

## Full Documentation

For comprehensive scenario library:
- `docs/testing/SCENARIOS.md` — All test scenarios with results
- `docs/testing/TESTING_GUIDE.md` — Complete API reference and troubleshooting