Testing API Manual

Manual API Testing for Pharmaceutical Test Generation

Overview

This skill provides verified WSL commands for manual testing of the pharmaceutical test generation API. All commands have been validated on WSL2 with Docker Desktop on ARM64 (Qualcomm Oryon) hardware.

Verified Status: ✅ Production-ready (WSL2 + Docker Desktop, ARM64 compatible)

When to Use This Skill

• Testing Docker-based FastAPI endpoints manually
• Submitting Category 3 URS files to avoid human-in-the-loop consultation
• Monitoring asynchronous job execution in real-time
• Debugging API authentication issues with Clerk JWT
• Validating Task 3.6 fixes (test_suite serialization, retry logic, ALCOA+ logs)
• Inspecting container logs for workflow debugging

Prerequisites

Before testing, verify:

1. Docker containers running:

   docker ps
   # Expected: pharma-api-dev, pharma-worker-dev, pharma-postgres-dev, pharma-localstack-dev
   

2. API accessible on port 8080:

   curl http://localhost:8080/health
   # Expected: {"status":"healthy"}
   

3. .env.local with Clerk credentials:

   • CLERK_SECRET_KEY
   • CLERK_ISSUER
   • CLERK_PEM_PUBLIC_KEY

4. Token generation script: scripts/get_clerk_token.py

5. Test URS files available: datasets/urs_corpus_v2/category_3/URS-020.md

Complete Testing Workflow

Step 1: Generate Clerk JWT Token

⚠️ CRITICAL: Use python3 on WSL2, not python.

# Navigate to project directory
cd /mnt/c/Users/anteb/Desktop/Courses/Projects/thesis_project

# Generate and save token in variable
TOKEN=$(python3 scripts/get_clerk_token.py --user-id user_35KgiAcvIC0tdtFvJUN1vDkrNYc --env-file .env.local)

# Verify token is stored (optional)
echo "Token stored: ${TOKEN:0:50}..."

Expected Output:

Token stored: eyJhbGciOiJSUzI1NiIsImNhdCI6ImNsX0I3ZDRQRDIyMk...

Token Expiry: Clerk session tokens expire after 60 seconds. Regenerate if needed.

Step 2: Submit URS Job to API

Why Category 3? Category 3 (non-configured products) skips human-in-the-loop consultation, enabling faster end-to-end testing.

# Submit URS job (Category 3 to avoid human consultation)
curl -X POST http://localhost:8080/jobs \
  -H "Authorization: Bearer $TOKEN" \
  -F "file=@datasets/urs_corpus_v2/category_3/URS-020.md"

Expected Success Response:

{
  "job_id": "67077789-b62b-4751-a475-7ddf77d30708",
  "status": "pending",
  "message": "Job submitted successfully"
}

Common Errors:

Step 3: Save Job ID and Monitor Status

One-Time Check:

# Save job_id from Step 2 response
JOB_ID="67077789-b62b-4751-a475-7ddf77d30708"  # Replace with actual job_id

# Check job status
curl http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN"

Continuous Monitoring Loop:

# Monitor status every 10 seconds (requires jq)
while true; do
  curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.status, .gamp_category'
  sleep 10
done

Expected Status Transitions:

"pending"    →  "processing"  →  "completed"
null         →  "CATEGORY_3"  →  "CATEGORY_3"

Stop Loop: Press Ctrl+C

Install jq (if not available):

sudo apt-get update && sudo apt-get install -y jq

Step 4: Watch Container Logs in Real-Time

API Container Logs:

docker logs pharma-api-dev -f --tail=50

Worker Container Logs (Workflow Execution):

docker logs pharma-worker-dev -f --tail=50

Stop Logs: Press Ctrl+C

What to Look For:

• ✅ INFO: GAMP-5 Category 3 detected - Categorization succeeded
• ✅ INFO: ChromaDB query returned 182 chunks - RAG retrieval working
• ✅ INFO: Parallel agents completed - Research/SME agents finished
• ✅ INFO: Serialized test suite to YAML - Test suite generated (Task 3.6 fix)
• ✅ INFO: ALCOA+ audit record persisted - Audit logs writing (Task 3.6 fix)
• ❌ ERROR: ModuleNotFoundError: langfuse.decorators - Langfuse import error (should be fixed)
• ❌ WARNING: Job retry 1/3 - Retry loop starting (should stop after 3 retries per Task 3.6)

Step 5: Retrieve Job Results

Get Complete Job Details:

curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.'

Extract Specific Fields:

# Status only
curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.status'

# GAMP category
curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.gamp_category'

# Test suite (if available in response)
curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.result.test_suite'

# Error message (if failed)
curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.error'

Complete Testing Session Example

# 1. Generate token
TOKEN=$(python3 scripts/get_clerk_token.py --user-id user_35KgiAcvIC0tdtFvJUN1vDkrNYc --env-file .env.local)
echo "Token: ${TOKEN:0:30}..."

# 2. Submit job
RESPONSE=$(curl -s -X POST http://localhost:8080/jobs \
  -H "Authorization: Bearer $TOKEN" \
  -F "file=@datasets/urs_corpus_v2/category_3/URS-020.md")

echo "Response: $RESPONSE"

# 3. Extract job_id
JOB_ID=$(echo "$RESPONSE" | jq -r '.job_id')
echo "Job ID: $JOB_ID"

# 4. Monitor in loop (Ctrl+C to stop)
while true; do
  STATUS=$(curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq -r '.status')
  echo "[$(date +%H:%M:%S)] Status: $STATUS"

  if [ "$STATUS" = "completed" ] || [ "$STATUS" = "failed" ]; then
    echo "Job finished with status: $STATUS"
    break
  fi

  sleep 10
done

# 5. Get final results
curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.'

Task 3.6 Validation Checklist

Use these commands to verify all Task 3.6 fixes work correctly:

✅ Fix 1: Test Suite YAML Serialization

# Check if test_suite key present in workflow result
curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.result.test_suite' | head -20

# Expected: YAML content, NOT null

✅ Fix 2: consultation_result Optional (Category 3)

# Watch worker logs for consultation errors
docker logs pharma-worker-dev --tail=100 2>&1 | grep -i "consultation_result"

# Expected: NO errors about missing consultation_result for Category 3

✅ Fix 3: Infinite Retry Loop Fixed

# Watch for retry pattern (should stop after 3-4 retries)
docker logs pharma-worker-dev -f 2>&1 | grep -i "retry"

# Expected: Max 3-4 retries, then STOP (no infinite loop)

✅ Fix 4: ALCOA+ Audit Logs Persist

# Check if audit logs written to host filesystem
ls -lh main/logs/audit/alcoa_records_*.json

# Expected: Files present with recent timestamps

✅ Fix 5: Langfuse Instrumentation

# Check for Langfuse import errors
docker logs pharma-api-dev --tail=200 2>&1 | grep -i "langfuse"
docker logs pharma-worker-dev --tail=200 2>&1 | grep -i "langfuse"

# Expected: NO ModuleNotFoundError: langfuse.decorators

Troubleshooting

Issue: Token Authentication Fails

Symptoms:

{"detail":"Not authenticated"}

Solutions:

1. Regenerate token (expires after 60 seconds):

   TOKEN=$(python3 scripts/get_clerk_token.py --user-id user_35KgiAcvIC0tdtFvJUN1vDkrNYc --env-file .env.local)
   

2. Verify token format:

   echo "$TOKEN" | cut -d'.' -f1 | base64 -d 2>/dev/null | jq '.'
   # Should show JWT header with "alg":"RS256"
   

3. Check Clerk credentials in .env.local:

   grep CLERK .env.local
   

Issue: Containers Not Running

Symptoms:

curl: (7) Failed to connect to localhost port 8080

Solutions:

1. Check container status:

   docker ps --format "table {{.Names}}\t{{.Status}}"
   

2. Restart containers:

   docker-compose -f docker-compose.dev.yml down
   docker-compose -f docker-compose.dev.yml up -d
   

3. Check container logs for errors:

   docker logs pharma-api-dev --tail=50
   docker logs pharma-worker-dev --tail=50
   

Issue: File Not Found

Symptoms:

curl: (26) Failed to open/read local data from file/application

Solutions:

1. Verify file exists:

   ls -lh datasets/urs_corpus_v2/category_3/URS-020.md
   

2. Use correct relative path from project root:

   pwd  # Should be: /mnt/c/Users/anteb/Desktop/Courses/Projects/thesis_project
   

3. Try alternative URS files:

   ls datasets/urs_corpus_v2/category_3/
   # Use any URS-*.md file
   

Issue: Job Stuck in Processing

Symptoms:

• Status never changes from "processing"
• Worker logs show errors or infinite retries

Solutions:

1. Check worker logs:

   docker logs pharma-worker-dev -f
   

2. Look for specific errors:

   • ModuleNotFoundError → Langfuse import error (Task 3.6 fix needed)
   • WARNING: Job retry X/3 → Workflow failing, check error message
   • ERROR: Context retrieval failed for key consultation_result → Task 3.6 fix needed

3. Check job error message:

   curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.error'
   

Environment Information

Tested On:

• OS: Windows 11 with WSL2 (Ubuntu)
• CPU: Qualcomm Oryon (ARM 64-bit)
• Docker: Docker Desktop for Windows (ARM64)
• Python: 3.12
• FastAPI Port: 8080 (NOT 8000)

Key Differences from Native Linux:

• Use python3 instead of python
• File paths use /mnt/c/Users/... for Windows drives
• Docker Desktop runs containers via WSL2 backend

Quick Reference Commands

# Token generation
TOKEN=$(python3 scripts/get_clerk_token.py --user-id user_35KgiAcvIC0tdtFvJUN1vDkrNYc --env-file .env.local)

# Job submission
curl -X POST http://localhost:8080/jobs -H "Authorization: Bearer $TOKEN" -F "file=@datasets/urs_corpus_v2/category_3/URS-020.md"

# Status check
curl http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN"

# Monitor loop
while true; do curl -s http://localhost:8080/jobs/$JOB_ID -H "Authorization: Bearer $TOKEN" | jq '.status, .gamp_category'; sleep 10; done

# Container logs
docker logs pharma-api-dev -f --tail=50
docker logs pharma-worker-dev -f --tail=50

# Container status
docker ps --format "table {{.Names}}\t{{.Status}}"

# Restart containers
docker-compose -f docker-compose.dev.yml restart

# Check audit logs
ls -lh main/logs/audit/alcoa_records_*.json

Success Criteria

A successful test execution should show:

1. ✅ Token generated without errors
2. ✅ Job submitted with job_id returned
3. ✅ Status transitions: pending → processing → completed
4. ✅ GAMP category detected: CATEGORY_3
5. ✅ Test suite present in result (Task 3.6 fix validation)
6. ✅ No consultation_result errors (Task 3.6 fix validation)
7. ✅ Max 3-4 retries on failure (Task 3.6 fix validation)
8. ✅ ALCOA+ audit logs persisted to filesystem (Task 3.6 fix validation)
9. ✅ No Langfuse import errors in logs (Task 3.6 fix validation)

Estimated Duration: 5-10 minutes for complete workflow execution

Related Skills

• testing-api-authentication - Automated API authentication testing
• clerk-token-ops - Clerk JWT token operations and validation
• langfuse-integration - Langfuse observability setup

Created: 2025-11-17 Last Updated: 2025-11-17 Status: Verified and Production-Ready